BestHub
Sign in
Tag

pretrained language model

0 views collected around this technical thread.

DataFunTalk
DataFunTalk
Jan 30, 2023 · Artificial Intelligence

Domain Knowledge Enhanced Pretrained Language Model for Medicinal Product Vertical Search

This article presents a domain‑knowledge‑enhanced pretrained language model that combines ELECTRA‑based token‑level masking with a novel product‑attribute prediction (PAP) task to improve query understanding, intent classification, and relevance matching in vertical drug e‑commerce search, and validates its effectiveness through extensive experiments on public and proprietary datasets.

ELECTRAdomain knowledgemedical NLP
0 likes · 13 min read
Domain Knowledge Enhanced Pretrained Language Model for Medicinal Product Vertical Search
DataFunTalk
DataFunTalk
Jun 20, 2022 · Artificial Intelligence

SMedBERT: Knowledge‑Enhanced Pre‑trained Language Model for Medical Text Mining and Its Business Applications

The article introduces Dingxiangyuan's medical knowledge‑graph ecosystem, describes the construction of a four‑layer taxonomy, presents the ACL‑published SMedBERT model that injects structured medical semantics into a pre‑trained language model, and discusses its deployment in search, query expansion, and semantic matching while outlining future challenges.

Healthcare AIKnowledge GraphSMedBERT
0 likes · 13 min read
SMedBERT: Knowledge‑Enhanced Pre‑trained Language Model for Medical Text Mining and Its Business Applications
DataFunTalk
DataFunTalk
Jun 16, 2022 · Artificial Intelligence

BigBang Transformer (BBT): A 1‑Billion‑Parameter Financial Pre‑trained Language Model with Time‑Series‑Text Cross‑Modal Architecture

The BigBang Transformer (BBT) is a 1‑billion‑parameter financial pre‑trained language model that combines text and time‑series data in a cross‑modal Transformer architecture, achieving up to 10% higher downstream accuracy than T5‑scale models and demonstrating strong performance on financial NLP tasks, time‑series forecasting, and multi‑factor investment strategies.

artificial intelligencebig datacross‑modal
0 likes · 19 min read
BigBang Transformer (BBT): A 1‑Billion‑Parameter Financial Pre‑trained Language Model with Time‑Series‑Text Cross‑Modal Architecture
Kuaishou Tech
Kuaishou Tech
Nov 16, 2021 · Artificial Intelligence

KuaiSearch's PERKS Pre‑trained Language Model Sets New Record on the CLUE Benchmark

The KuaiSearch research team introduced PERKS, a large‑scale Chinese pre‑trained language model that achieved an 80.618 score on the CLUE 1.1 language classification task, narrowing the gap to human annotation and demonstrating significant advances in multi‑stage training, model optimization, and real‑world search applications.

CLUE benchmarkKuaiSearchNLP
0 likes · 7 min read
KuaiSearch's PERKS Pre‑trained Language Model Sets New Record on the CLUE Benchmark
DataFunTalk
DataFunTalk
Oct 24, 2020 · Artificial Intelligence

FinBERT 1.0: An Open‑Source Chinese Financial Domain Pre‑trained BERT Model and Its Evaluation

FinBERT 1.0 is an open‑source Chinese BERT model pre‑trained on large‑scale financial corpora that achieves 2‑5 % F1 improvements across multiple downstream fintech tasks without additional tuning, demonstrating the value of domain‑specific pre‑training for natural language processing.

BERTDeep LearningFinBERT
0 likes · 14 min read
FinBERT 1.0: An Open‑Source Chinese Financial Domain Pre‑trained BERT Model and Its Evaluation
58 Tech
58 Tech
Jun 5, 2020 · Artificial Intelligence

qa_match V1.1: Upgraded Lightweight Deep Learning QA Matching Tool

The article introduces qa_match V1.1, an open‑source, Apache‑licensed lightweight question‑answer matching system that adds a simple pre‑trained language model (SPTM), supports one‑level knowledge bases, details model architecture, training resources, performance benchmarks, future plans, and contribution guidelines.

AIDeep Learningknowledge base
0 likes · 10 min read
qa_match V1.1: Upgraded Lightweight Deep Learning QA Matching Tool