Conditional and Multimodal Knowledge Graph Construction, Extraction, and Integration with Large Models

The talk introduces conditional knowledge graphs, which emphasize temporal and contextual constraints, and multimodal knowledge graphs that fuse text, images, and other modalities to enrich information representation.

It outlines five main topics: (1) background of knowledge graphs, (2) construction of conditional knowledge graphs, (3) construction of multimodal knowledge graphs, (4) integration of knowledge graphs with large models, and (5) a Q&A session.

For conditional KG construction, the authors discuss fact‑triplet extraction, closed‑ vs. open‑domain extraction, and the need to capture contradictory or context‑dependent facts, proposing a four‑layer graph structure (entity, relation, condition, and order layers).

To support research, a specially annotated dataset containing fact and condition triples (and extended five‑tuple forms) is released, enabling supervised training of extraction models.

Two extraction pipelines are described: a basic staged approach (relation extraction → entity completion → correspondence checking) and a multi‑input/multi‑output approach that combines outputs from language models, POS taggers, and key‑phrase detectors to resolve overlapping triples.

A semi‑supervised bootstrap method is introduced to enlarge training data by iteratively correcting model‑generated annotations using heuristic rules.

The integration with large models is explored through three stages: (i) knowledge‑graph‑augmented input (e.g., K‑Bert), (ii) representation‑learning constraints derived from conditional KG, and (iii) mutual distillation between fact‑aware and condition‑aware models.

Finally, the Q&A addresses how large models can dynamically update conditional KG, control the knowledge they output, and apply conditional KG‑enhanced models to domains such as intelligent medical decision support.