Helping to Diagnose 362 Common Diseases! Cambridge/Oxford/Warwick University and Others Proposed a Multi-agent Large Language Model Framework to Automatically Build a Medical Knowledge Graph
Today, the shortage of medical resources is a long-term problem plaguing the global medical system, and the "doctor shortage" is particularly prominent in the fields of primary care and general medicine. The World Health Organization predicts that by 2030, there will be a shortage of 15 million medical workers worldwide. In China, according to data from the National Bureau of Statistics and the National Health Commission, although the overall number of doctors in China has increased year by year, the number of doctors in some remote and rural areas is far from enough.
Tiered diagnosis and treatment is one of the effective models to alleviate the "doctor shortage". Under this model, when a patient arrives at the hospital, if he or she is unable to determine the problem, the general practitioner will first conduct a preliminary examination to determine the general direction of the problem, and then refer the patient to a relevant specialist, such as cardiology, neurology, etc., for further diagnosis and treatment. This hierarchical and collaborative model ensures the efficiency, comprehensiveness and professionalism of diagnosis, reduces the workload of specialists, makes their training more targeted, shortens the training cycle, and quickly replenishes doctor resources.
However, doctors have limited time and energy after all. How to use advanced technologies such as artificial intelligence to improve diagnostic efficiency and thus alleviate the shortage of doctor resources has become a focus of the industry. Inspired by the above-mentioned medical stratified collaboration model,A research team from the University of Warwick, Cranfield University, University of Cambridge and University of Oxford proposed KG4Diagnosis.This is a new hierarchical multi-agent framework that simulates the real-world medical system of collaboration between general practitioners and specialists through a two-layer architecture. It is used to automate the construction, diagnosis, treatment and reasoning of medical knowledge graphs. It overcomes the limitations of existing medical knowledge graph construction methods in scalability and processing of unstructured data, and covers 362 common diseases, spanning multiple medical fields such as obesity.
The related results were published in the top conference AAAI-25 Bridge Program under the title "KG4Diagnosis: A Hierarchical Multi-Agent LLM Framework with Knowledge Graph Enhancement for Medical Diagnosis".
Follow the official account and reply "Medical Knowledge Graph" to get the complete PDF
MedQA Medical Text Question Answering Dataset:
The open source project "awesome-ai4s" brings together more than 100 AI4S paper interpretations and provides massive data sets and tools:
https://github.com/hyperai/awesome-ai4s
What is the disruptive significance of KG4Diagnosis?
Why did the researchers develop KG4Diagnosis?
This actually stems from an idea, that is, whether artificial intelligence can be used to simulate the hierarchical medical system in the real world to solve the complexity of medical diagnostic reasoning. Thus, the KG4Diagnosis framework was born. This is a hierarchical multi-agent framework that integrates a general practitioner large language model (GPLLM) and multiple domain-specific expert large language models (Consultant-LLMs) - the general practitioner (GP) agent performs preliminary assessment and triage, and then coordinates with the specialist agent to perform domain-specific analysis.
How was the framework constructed?
In simple terms, it can be divided into 5 steps:
(1) The input medical text is segmented into data blocks that conform to the context constraints of the knowledge graph and processed by the semantic-based entity extraction and relation extraction modules;
(2) The extracted entities and relations are stored in a dedicated database;
(3) These databases are used to build the Medical Knowledge Graph;
(4) Combining medical knowledge graphs with large language models (LLMs) and multi-agent systems to enhance diagnostic reasoning;
(5) The diagnostic results are fed back to the user endpoint through human-guided reasoning.
Obviously, this method combines the broad capabilities of large language models with the precision of specialized medical knowledge to achieve more advanced diagnostic results. According to the research team, the system's training method covers 362 common diseases across multiple medical specialties, representing a wide range of medical diagnoses.
What is the actual application effect of this system?
The following diagram shows how the new framework proposed by the study collaborates on the diagnostic process and how AI systems can help provide personalized medical advice: the patient describes the symptoms, the doctor asks clarifying questions, and the AI provides explanations and suggestions.
* Patient: "Doc, I've been experiencing ringing in my ears lately, especially at night. It feels like there's a constant buzzing sound that affects my sleep and concentration."
* Doctor: “Is your tinnitus constant or does it come and go? Do you have other symptoms, such as headaches, hearing loss, or ear pain?”
* AI: "I'm sorry to hear about your discomfort. Tinnitus can be caused by a variety of factors, such as exposure to loud noise, ear infections, high blood pressure or stress. Have you had a hearing test or ear exam recently? These exams can help determine the severity and possible causes of your tinnitus."
The figure below further demonstrates the expertise of the system in the knowledge graph in the field of obesity, showing the relationship between obesity, type 2 diabetes and cardiovascular disease, their common symptoms, treatments and comorbidities, and emphasizing the multifaceted role of drugs in solving complex health problems.
In summary, this study solved the hallucination challenge faced by large language models (LLMs) in medical diagnosis through multi-agent verification and knowledge graph constraints, and verified the effectiveness of these mechanisms through comprehensive benchmark tests. At the same time, the researchers also demonstrated the practical value of the framework through real-world medical scenarios, and its future applications are worth looking forward to.
Technology and application blossom! Artificial intelligence-assisted diagnosis has a promising future
Research progress of medical knowledge graph
KG4Diagnosis proposed by the above research team is a hierarchical multi-agent large language model framework based on knowledge graph enhancement. The so-called knowledge graph (KGs) is a symbolic description of concepts in the physical world and their relationships.
Currently, methods for constructing medical knowledge graphs include traditional rule-based systems and advanced artificial intelligence models. Rule-based methods provide reliability but lack scalability, while language models like GPT and MedPaLM show potential in generating structured knowledge from unstructured data, but they face problems of illusion and accuracy.
Faced with these challenges, many research teams have explored in multiple directions: for example, in order to solve the challenge that large language models are prone to hallucinations, retrieving relevant facts from knowledge graphs is considered a promising approach. Existing KG enhancement methods often require multiple rounds of retrieval and verification of each fact, which hinders their application in practical scenarios. A research team from Imperial College London proposed a self-improvement enhanced knowledge graph retrieval (Re-KGR) method to enhance the factuality of LLMs responses in the medical field with less retrieval work. The related results were published on arXiv under the title "Mitigating Hallucinations in Large Language Models via Self-Refinement-Enhanced Knowledge Retrieval".
At the same time, the industry has also achieved breakthrough results in building large medical models using medical knowledge graphs. For example, basic models (FMs) can learn from a large amount of unlabeled data to demonstrate excellent performance in a wide range of tasks. However, FMs developed for the biomedical field are still largely unimodal. To overcome this limitation, a research team from UIUC proposed BioBRIDGE, which uses knowledge graphs to learn the conversion between one unimodal FM and another unimodal FM without fine-tuning any underlying unimodal FM.
The research results show that BioBRIDGE can beat the best baseline KG embedding method (on average about 76.3% higher) in cross-modal retrieval tasks. The related research results are titled "BIOBRIDGE: BRIDGING BIOMEDICAL FOUNDATION MODELS VIA KNOWLEDGE GRAPHS" and were accepted by ICLR 2024.
Artificial intelligence-assisted diagnosis helps improve efficiency and alleviate the shortage of doctor resources
The ultimate goal of technological progress is to feed back into practical applications, reduce doctors' workload, and improve the accuracy and efficiency of diagnosis. By integrating a large amount of medical knowledge and clinical data, AI has made many key advances in auxiliary diagnosis in recent years.
In November 2024, AI-assisted diagnosis was included in the National Health Insurance Administration's project establishment guidelines for the first time. In order to support the relatively mature AI-assisted technology to enter clinical applications and prevent additional burdens on patients, the National Health Insurance Administration analyzed the potential application scenarios of AI and established "AI-assisted" extensions in radiology, ultrasound, and rehabilitation projects. In a sense, this has laid a policy foundation for the large-scale application of AI-assisted diagnosis in the future.
In terms of specific applications, in February 2024, the South China University of Technology team announced that it had developed an intelligent eye disease screening system, which integrates an AI model trained based on a data set and a series of hardware devices such as a fundus camera. It can provide diagnostic recommendations for more than 30 diseases within 15 seconds after completing the fundus color photo shooting.
In June 2024, OpenAI stated in its official blog that Color Health used OpenAI's GPT-4o model to develop an AI assistant copilot to help doctors screen and treat cancer patients, thereby achieving better treatment efficiency. According to official data, medical institutions that use AI assistants can identify 4 times more pathological results than those that do not use AI assistants. In addition, after using the AI assistant, clinicians only need an average of 5 minutes to analyze patient records, while without it, the data will become fragmented, which may cause weeks of delays.
In addition, it is reported that iFlytek Medical has corrected 1.6 million major misdiagnoses and detected 60 million irrational or suspected erroneous medications through AI-assisted diagnosis, improving primary diagnosis and treatment capabilities and preventing misdiagnosis and missed diagnoses.
Based on technology, I believe that AI will become the "right-hand man" of doctors in the future, continuously improving the accessibility and quality of medical services.
References:
1.https://arxiv.org/abs/2412.16833
2.https://zh.wikipedia.org/wiki/%E7%9F%A5%E8%AD%98%E5%9C%96%E8%AD%9C
3.https://53ai.com/news/LargeLanguageModel/2024060240513.html
4.https://hub.baai.ac.cn/view/36889
5.https://www.yicai.com/news/102371268.html
6.https://m.yicai.com/news/102153745.html