Practical NER Techniques for Business Chatbots on the 58.com Service Platform

In the C‑end user and B‑end merchant IM chat scenario of 58.com, a smart chat assistant was built to automatically capture business opportunities such as phone numbers, addresses, times, and service objects. The talk focuses on the NER technology practice that enables this capability.

Background : 58.com is a large classified information platform connecting millions of users and merchants. Manual IM communication is costly, prompting the development of an AI‑driven chatbot.

Smart Chat Assistant : Since 2017, 58.com’s intelligent customer service system has been deployed across various business lines. In 2019 it was extended to the yellow‑page IM channel, where the assistant extracts key entities from user messages and forwards them to merchants, improving lead conversion.

Task‑oriented Dialogue Architecture : The system consists of NLU, Dialogue Management (DM), and NLG. DM merges state tracking and policy. A diagram of the architecture (omitted) shows the flow of a single interaction.

NER Model Selection : After accumulating labeled data, three deep‑learning NER models were evaluated – BiLSTM‑CRF, IDCNN‑CRF, and BERT. Experiments showed IDCNN‑CRF offered a good trade‑off between accuracy and latency, and was chosen for production.

Model Optimizations :

Data augmentation (label‑consistent token replacement and synonym replacement) to alleviate data scarcity and class imbalance.

Focal loss to address the overwhelming O‑label distribution in BIO tagging.

Fusion of rule‑based/lexicon methods with neural models to leverage high‑precision patterns for core entities (e.g., phone, WeChat) while using the model for more ambiguous entities.

Contextual modeling using bidirectional GRU to incorporate dialogue history, improving recognition of entities that depend on previous turns.

Online Results and Future Work : The fusion strategy raised F1 scores for core and rich entities by 1.65% and 1.33% respectively. Remaining challenges include rapid onboarding of new entity types and handling descriptive entities. Planned improvements involve few‑shot learning, a dedicated NER data‑augmentation method (DAGA), information‑extraction techniques, and experiments with newer pretrained models such as RoBERTa and DistilBERT.

In summary, the case study demonstrates how a production‑grade NER pipeline can be iteratively built, optimized, and deployed within a large‑scale commercial dialogue system.