A Review of Medical AI Breakthroughs in 2024, 35 Cutting-edge Papers You Can’t Miss
In the past few years, technology giants led by NVIDIA and Google have expressed their attention to AI healthcare, and nearly 100 billion yuan of funds have been invested in this field. The medical field has also become one of the areas where AI is most widely used and has the most obvious results.
In the year 2024 that is about to pass, researchers built large medical models, used AI to segment medical images/videos, diagnose diabetes, Parkinson's disease, breast cancer, lung cancer, ovarian cancer, coronary heart disease, depression, and stomach disease, while exploring deep learning technology for identifying RNA viruses. AI is reshaping the healthcare industry at an unprecedented pace and optimizing patients' medical experience.
Following the summary of 26 most noteworthy AI+materials chemistry papers in the previous issue,In this article, HyperAI focuses on the research of AI in the field of medical health, and has selected 35 cutting-edge papers interpreted from 2023 to 2024. Click on the paper title or Chinese interpretation below to jump to the paper interpretation page. I hope it will be helpful to you.
01 Paper Title:Unified Medical Image Pre-training in Language-Guided Common Semantic Space, 2024.07
Chinese interpretation:Selected for ECCV 2024! Zhejiang University and Microsoft Research Asia jointly proposed the unified medical image pre-training framework UniMedI, breaking the barriers of medical data heterogeneity
Research content:Zhejiang University and Microsoft Research Asia jointly proposed a new unified medical image pre-training framework UniMedI. It uses diagnostic reports as a common semantic space to create a unified representation for medical images of different modalities, successfully integrating 2D and 3D images, and making better use of complex medical data.
Chinese interpretation:The world's first! Feng Jianfeng's team at Fudan University developed a digital twin brain platform with 86 billion neurons
Research content:The Institute of Brain-Inspired Intelligence Science and Technology of Fudan University released the digital twin brain platform, which is the world's first full-human brain-scale brain simulation platform developed based on data assimilation methods, with 86 billion neurons and one trillion synapses.
03 Paper Title:Towards building multilingual language model for medicine, 2024.09
Chinese interpretation:The benchmark test in the medical field surpasses Llama 3 and is close to GPT-4. The Shanghai Jiaotong University team released a multilingual medical model covering 6 languages
Research content:The team from Shanghai Jiao Tong University created a multilingual medical corpus MMedC containing 25.5 billion tokens, developed a multilingual medical question-and-answer evaluation standard MMedBench covering 6 languages, and also built an 8B base model MMed-Llama 3.
04 Paper Title:Integrated image-based deep learning and language models for primary diabetes care, 2024.07
Chinese interpretation:The world's first! Tsinghua University/Shanghai Jiaotong University and others jointly built a visual-language model for diabetes diagnosis and treatment, published in Nature
Research content:Tsinghua University, in collaboration with Shanghai Jiao Tong University, the National University of Singapore and the Singapore National Eye Centre team, has successfully built the world's first integrated vision-large language model system, DeepDR-LLM, for diabetes diagnosis and treatment. This system can provide primary care doctors with personalized diabetes management advice and auxiliary diagnosis results for diabetic retinopathy.
05 Paper Title:Harnessing TME depicted by histological images to improve cancer prognosis through a deep learning system, 2024.05
Chinese interpretation:Directly attacking three major solid tumors! Shanghai Jiaotong University team released a deep learning system to improve the accuracy of cancer survival prediction
Research content:A team from Shanghai Jiao Tong University developed a deep learning system, IGI-DL, which uses histopathological images to predict tumor microenvironment information for cancer patients who do not have spatial transcriptome data, thereby achieving accurate cancer prognosis.
Chinese interpretation:Blood routine tests, urine tests and other indicators can identify ovarian cancer! Liu Jihong's team from Sun Yat-sen University led the team, and four major medical schools jointly built an AI fusion model
Research content:The gynecology team of Sun Yat-sen University Cancer Center, in collaboration with Southern Medical University, Tongji Hospital affiliated to Tongji Medical College of Huazhong University of Science and Technology, and Obstetrics and Gynecology Hospital affiliated to Zhejiang University School of Medicine, constructed the MCF artificial intelligence fusion model for ovarian cancer diagnosis. The model's accuracy in identifying ovarian cancer is better than traditional biomarkers such as CA125 and HE4.
07 Paper Title:Depression Diagnosis Dialogue Simulation: Self-improving Psychiatrist with Tertiary Memory, 2024.09
Chinese interpretation:Agent Psychological Clinic is online! Based on 1.3K depression consultation dialogues, the Shanghai Jiaotong University team built a large model dialogue agent that can diagnose depression
Research content:The X-LANCE Laboratory team of Shanghai Jiao Tong University and others have built an automated large-model dialogue agent simulation system - the Agent Mental Clinic AMC (Agent Mental Clinic) for the preliminary diagnosis of depression.
08 Paper Title:Medical SAM 2: Segment medical images as video via Segment Anything Model 2, 2024.08
Chinese interpretation:SAM 2's latest application has landed! The Oxford University team released Medical SAM 2, refreshing the SOTA list of medical image segmentation
Research content:The Oxford University team developed the Medical SAM 2 (MedSAM-2) medical image segmentation model, which is based on the SAM 2 framework design and treats medical images as videos. It not only performs well in 3D medical image segmentation tasks, but also unlocks a new single-prompt segmentation capability.
09 Paper Title:MemSAM: Taming Segment Anything Model for Echocardiography Video Segmentation, 2024.05
Chinese interpretation:Candidate for CVPR 2024 Best Paper! Shenzhen University and Hong Kong Polytechnic University jointly released MemSAM: Applying the "Segment Everything" model to medical video segmentation
Research content:Shenzhen University and the Intelligent Health Research Center of the Hong Kong Polytechnic University jointly proposed a novel echocardiography video segmentation model MemSAM, applying SAM to medical videos.
Chinese interpretation:Huazhong University of Science and Technology proposes a medical image segmentation model for ultra-large-scale pathological image analysis to improve the accuracy of Sjögren's syndrome diagnosis
Research content:The team from Huazhong University of Science and Technology proposed the medical image segmentation model M2CF-Net. By integrating multi-resolution and multi-scale image recognition technologies, this method can accurately identify lymphocyte aggregation foci in pathological images of patients with Sjögren's syndrome, helping doctors make faster and more accurate diagnoses.
11 Paper Title:S2P-Matching: Self-supervised Patch-based Matching Using Transformer for Capsule Endoscopic Images Stitching, 2024.09
Chinese interpretation:The matching accuracy rate increased by 187.9%! The CGCL laboratory of Huazhong University of Science and Technology uses self-supervised learning to assist capsule endoscopy image stitching, and the "Sky Eye" can also see gastrointestinal health
Research content:Huazhong University of Science and Technology, in collaboration with teams from Shanghai Jiao Tong University, South-Central University for Nationalities, Hong Kong University of Science and Technology, Hong Kong Polytechnic University, and the University of Sydney, proposed a self-supervised, fragment-matching-based capsule endoscopy image stitching method, S2P-Matching, for the early diagnosis of gastrointestinal diseases.
Chinese interpretation:Tsinghua team proposes AI-based model ROAM to achieve accurate diagnosis of glioma
Research content:Tsinghua University, in collaboration with Xiangya Hospital of Central South University, has proposed a basic AI model for precise pathological diagnosis, ROAM, based on large regional interests and pyramid Transformer. It is used for clinical-level diagnosis and molecular marker discovery of gliomas and can be extended to pathological diagnosis of other types of tumors.
13 Paper Title:Large-scale pancreatic cancer detection via noncontrast CT and deep learning, 2023.11
Chinese interpretation:31 cases of missed diagnosis were identified among 20,000 cases. Alibaba Damo Academy took the lead in launching "plain scan CT + large model" to screen pancreatic cancer
Research content:Alibaba DAMO Academy, in collaboration with more than a dozen medical institutions at home and abroad, released the PANDA large model to achieve early screening for pancreatic cancer, and discovered 31 clinically missed lesions in a real-world consecutive patient population of more than 20,000.
14 Paper Title:CGS-Mask: Making Time Series Predictions Intuitive for All, 2024.03
Chinese interpretation:Cracking the "black box" problem of time series prediction! Huazhong University of Science and Technology proposed CGS-Mask to reveal the key indicators of patient survival rate
Research content:Huazhong University of Science and Technology, in collaboration with the University of Sydney, Tongji Hospital, and others, proposed the CGS-Mask method, which is suitable for various time series forecasting tasks, especially those that require interaction with users and explanation of results, such as stock market forecasting, disease forecasting, and weather forecasting. It can not only improve the prediction accuracy of the model, but also increase the interpretability of the prediction results.
15 Paper Title:GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI, 2024.08
Chinese interpretation:Containing 284 data sets, covering 18 clinical tasks, Shanghai AI Lab and others released the multimodal medical benchmark GMAI-MMBench
Research content:The Shanghai Artificial Intelligence Laboratory, together with teams from the University of Washington/Monash University/East China Normal University, proposed the multimodal medical benchmark GMAI-MMBench, which includes 284 downstream task datasets from around the world. This dataset has been launched on the HyperAI official website!
16 Paper Title:Polyamine Anabolism Promotes Chemotherapy-Induced Breast Cancer Stem Cell Enrichment, 2024.07
Chinese interpretation:Fighting chemotherapy resistance and tumor recurrence! Shandong University research team uses AI to build a powerful defense line for breast cancer stem cells
Research content:Shandong University, in collaboration with research teams from Shanxi Medical University and Helix Matrix, used machine learning technology and mRNA analysis to successfully develop a new method, the BCSC signature, to assess the characteristics of cancer stem cells in samples from primary breast cancer patients, providing a new strategy and direction for the clinical treatment of breast cancer.
17 Paper Title:MlRS: An Al scoring system for predicting the prognosis and therapy of breast cancer, 2023.11
Chinese interpretation:Aiming at the world's most common cancer, Chinese scholars established the breast cancer prognostic scoring system MIRS
Research content:Researchers from the University of Kentucky, Macau University of Science and Technology, University of Macau, and the First Affiliated Hospital of Guangzhou Medical University used a neural network model to establish a scoring system MIRS for predicting breast cancer prognosis and treatment, which can be used to guide the formulation of treatment strategies for breast cancer patients.
18 Paper Title:A foundation model for generalizable disease detection from retinal images, 2023.08
Chinese interpretation:1.6 million+ unlabeled images, 3-dimensional comprehensive evaluation, Zhou Yukun and others developed the RETFound model to predict multiple systemic diseases using retinal images
Research content:Researchers from University College London (UCL) and Moorfields Eye Hospital proposed a retinal image foundation model, RETFound, which has excellent performance in tasks such as eye disease diagnosis/prognosis and prediction of systemic diseases.
19 Paper Title:A deep learning system for predicting time to progression of diabetic retinopathy, 2024.01
Chinese interpretation:Shanghai Jiao Tong University and Tsinghua University jointly released DeepDR Plus, which can predict the progression of diabetic retinopathy within 5 years using only fundus images
Research content:DeepDR Plus, jointly released by Shanghai Jiao Tong University, Tsinghua University and others, can predict the progression of diabetic retinopathy within 5 years based solely on fundus images.
20 Paper Title:Beneficial associations between outdoor visible greenness at the workplace and metabolic syndrome in Chinese adults, 2024.01
Chinese interpretation:More than 50,000 people participated in the study, and the team of Professor Wu Xifeng of Zhejiang University published a new study: Health is related to the level of greening in office spaces
Research content:The Zhejiang University team used a convolutional neural network model to evaluate visible green exposure based on the green view index of street view images, and confirmed that a higher green landscape index around the workplace is beneficial for adults to reduce the risk of metabolic syndrome.
21 Paper Title:ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image, 2024.07
Chinese interpretation:Selected for ECCV 2024! Covering 54,000+ images, MIT proposed a general model for medical image segmentation, ScribblePrompt, which performs better than SAM
Research content:A team from MIT's Computer Science and Artificial Intelligence Laboratory and others proposed a general model for interactive biomedical image segmentation, ScribblePrompt, which supports different annotation methods and can flexibly perform segmentation tasks, and can even be used for untrained labels and image types.
22 Paper Title:An interpretable model based on graph learning for diagnosis of Parkinson's disease with voice-related EEG, 2024.01
Chinese interpretation:The accuracy of early diagnosis of Parkinson's disease has been improved to 90.2%. Shenzhen Institute of Advanced Technology and Zhongshan First Hospital jointly proposed the GSP-GCNs model
Research content:A research team from the First Affiliated Hospital of Sun Yat-sen University and the Institute of Advanced Technology of USTC proposed a deep learning model - graph signal processing-graph convolutional networks (GSP-GCNs), which uses event-related EEG data to diagnose Parkinson's disease.
23 Paper Title:Pretraining a foundation model for generalizable fluorescence microscopy-based image restoration, 2024.04
Chinese interpretation:Collecting 30GB and nearly 200,000 pairs of training samples, the Fudan University team released UniFMIR: Using AI to break the limits of microscopic imaging
Research content:The Fudan University team proposed a cross-task, multi-dimensional image enhancement basic AI model UniFMIR, which broke through the existing limits of fluorescence microscopy imaging and provided a universal solution for fluorescence microscopy image enhancement.
24 Paper Title:Using artificial intelligence to document the hidden RNA virosphere, 2024.09
Chinese interpretation:AI helps RNA virus research achieve historic breakthroughs; Sun Yat-sen University and others use deep learning models to discover more than 160,000 new viruses
Research content:Sun Yat-sen University School of Medicine, in collaboration with Zhejiang University, Fudan University, China Agricultural University, City University of Hong Kong, Guangzhou University, University of Sydney, Alibaba Cloud Feitian Laboratory, etc., proposed a new deep learning model LucaProt, which discovered 180 supergroups and more than 160,000 new RNA viruses, and also discovered the longest RNA virus genome to date, marking a major breakthrough in the field of RNA virus identification.
25 Paper Title:Pianno: a probabilistic framework automating semantic annotation for spatial transcriptomics, 2024.04
Chinese interpretation:New achievement of Fudan Institute of Brain Science: Pianno, a spatial transcriptome semantic annotation tool, was developed based on semantic segmentation
Research content:The Fudan University team proposed the concept of "spatial transcriptome semantic annotation" and developed the spatial transcriptome semantic annotation tool Pianno, which can automatically define structures or cell types for spatial points within tissues, thereby combining information from multiple dimensions to enhance the interpretation of complex biological systems.
26 Paper Title:Health equity assessment of machine learning performance (HEAL): a framework and dermatology AI model case study, 2024.04
Chinese interpretation:Google releases HEAL framework, 4 steps to evaluate whether medical AI tools are fair
Research content:The Google team developed the HEAL (The health equity framework) framework, which can quantitatively evaluate whether machine learning-based healthcare solutions are "fair."
27 Paper Title:Assistive Al in Lung Cancer Screening: A Retrospective Multinational Study in the United States and Japan, 2024.03
Chinese interpretation:Based on clinical data from 627 patients in the United States and Japan, Google confirms the effectiveness of AI-assisted lung cancer screening in the population
Research content:The Google AI team developed and optimized the workflow for AI-assisted lung cancer screening and conducted multinational studies in the United States and Japan.
Chinese interpretation:Led by Peking Union Medical College Eye Hospital, five ophthalmology centers work together to use AI to assist in the detection of 13 types of fundus diseases
Research content:A joint research team from Peking Union Medical College Hospital, West China Hospital, the Second Hospital of Hebei Medical University, Tianjin Medical University Eye Hospital, and Wenzhou Medical University Eye Hospital developed an artificial intelligence system model to help junior ophthalmologists improve their diagnostic consistency by approximately 12%, providing a new method for the automatic detection of 13 major fundus diseases.
Chinese interpretation:By collecting data from 451 elderly patients with coronary heart disease at 301 Hospital, Hubei Macheng People's Hospital launched a machine learning model to accurately predict the mortality rate of patients within one year
Research content:Researchers from the People's Hospital of Macheng City, Hubei Province compared multiple models and used the best-performing machine learning model to predict the one-year mortality rate of elderly Chinese patients with coronary heart disease, diabetes, or impaired glucose tolerance to be 26.83%.
30 Paper Title:OBlA: An Open Biomedical Imaging Archive, 2023.08
Chinese interpretation:OBIA: 900+ patients, 193w+ images, the Chinese Academy of Sciences Institute of Genomics released my country's first biological image sharing database
Research content:The Institute of Genomics, Chinese Academy of Sciences (National Center for Bioinformation, China) has established the Open Biomedical Imaging Archive (OBIA), the first open repository of biomedical imaging data and related clinical data in China, which is open to medical practitioners and related scholars around the world free of charge.
31 Paper Title:A high-performance neuroprosthesis for speech decoding and avatar control, 2023.08
Chinese interpretation:A stroke left her speechless for 18 years, AI + brain-computer interface helps her “speak with thoughts”
Research content:A research team from the University of California, San Francisco and the University of California, Berkeley, has used AI to develop a new brain-computer technology that allows patients who have suffered aphasia for 18 years to "speak" again, and generates vivid facial expressions based on digital avatars, helping patients to communicate with others in real time at a speed and quality consistent with normal social interactions.
Chinese interpretation:Effectively delaying dementia: Yonsei University found that the gradient boosting machine model can accurately predict BPSD subsyndrome
Research content:Researchers from Yonsei University developed multiple machine learning models to predict BPSD, and experimental results showed that machine learning can effectively predict BPSD subsyndromes.
33 Paper Title:Robust Feature Selection strategy detects a panel of microRNAs as putative diagnostic biomarkers in Breast Cancer, 2023.07
Chinese interpretation:Feature selection strategy: finding new ways to detect breast cancer biomarkers
Research content:Researchers from the University of Naples Federico II in Italy proposed a feature selection strategy for detecting breast cancer biomarkers, and recommended that the 20 microRNAs they discovered be used as diagnostic biomarkers for breast cancer.
34 Paper Title:Performance of a Breast Cancer Detection Al Algorithm Using the Personal Performance in Mammographic Screening Scheme, 2023.09
Chinese interpretation:"Pink Killer" wanted poster, AI's ability to read breast X-rays is comparable to that of doctors
Research content:Researchers at the University of Nottingham in the UK compared the accuracy of commercial AI Lunit with that of doctors in reading mammograms. The results showed that Lunit's ability to analyze mammograms is comparable to that of human physicians.
35 Paper Title:Machine Learning-Enabled Tactile Sensor Design for Dynamic Touch Decoding, 2023.09
Chinese interpretation:Zhejiang University uses SVM to optimize tactile sensors, and the Braille recognition rate reaches 96.12%
Research content:Researchers from Zhejiang University have optimized the design of tactile sensors, which can accurately identify six dynamic touch patterns and can be used in health monitoring, intelligent robots, human-computer environment interaction, and virtual/augmented reality.
The above are the cutting-edge papers on AI+ healthcare summarized in this issue. For more latest results, please see: