Future Doctor, a China-based artificial intelligence healthcare technology company, has partnered with 32 clinical experts to publish groundbreaking research in Nature Portfolio’s npj Digital Medicine, introducing the “Clinical Safety-Effectiveness Dual-Track Benchmark” (CSEDB). This new framework aims to systematically evaluate whether medical AI systems are both safe and effective when used in real-world clinical settings—a critical step toward ensuring trustworthy AI in healthcare. The study addresses a growing concern in the field: while AI models have shown promise in medical applications, there remains a lack of standardized, rigorous benchmarks to assess their performance in actual clinical decision-making. The CSEDB framework evaluates AI systems across two key dimensions: safety and effectiveness. Safety refers to the model’s ability to avoid harmful or dangerous recommendations, such as incorrect diagnoses or inappropriate treatment suggestions. Effectiveness measures how well the AI supports accurate, timely, and clinically sound decisions. Unlike traditional benchmarks that focus only on accuracy or task completion, the CSEDB emphasizes real-world usability and risk mitigation, reflecting the high stakes involved in medical applications. The researchers applied the CSEDB to compare several leading large language models, including OpenAI’s o3, Google’s Gemini 2.5 Pro, and other prominent models. The evaluation involved real clinical scenarios, such as interpreting patient histories, recommending diagnostic tests, and suggesting treatment plans, using datasets from actual medical records. The results revealed significant disparities in both safety and effectiveness across models. While some models performed well in generating plausible responses, many failed to meet minimum safety thresholds—offering potentially dangerous or misleading advice in complex cases. Notably, the study found that models with strong general language capabilities did not necessarily translate into safe or accurate medical decisions. For example, some models confidently provided treatment suggestions without considering patient-specific contraindications or clinical guidelines. Others struggled with ambiguity, over-relying on patterns in training data rather than clinical reasoning. The research underscores the need for specialized evaluation frameworks tailored to healthcare, where errors can have life-threatening consequences. The CSEDB is designed to be adaptable across specialties and use cases, from radiology and cardiology to mental health and primary care. It also incorporates feedback from practicing clinicians, ensuring that the benchmark reflects real-world clinical workflows and priorities. The publication of this framework marks a pivotal moment in the development of medical AI. As AI systems increasingly support or even drive clinical decisions, the demand for transparent, reliable, and ethically sound evaluation methods grows. The CSEDB offers a practical, evidence-based approach to help regulators, developers, and healthcare providers assess AI tools before deployment. Future Doctor and its clinical collaborators are calling for broader adoption of the CSEDB by academic institutions, regulatory bodies, and AI developers. They argue that such benchmarks should become a standard requirement for any AI system intended for clinical use, similar to how pharmaceuticals undergo rigorous trials. The study also highlights the importance of continuous monitoring and updating of AI systems post-deployment, as medical knowledge evolves and new data emerges. The CSEDB framework includes mechanisms for ongoing evaluation, enabling real-time detection of performance degradation or emerging safety risks. With AI poised to transform healthcare delivery, this research provides a much-needed roadmap for ensuring that innovation does not come at the cost of patient safety. By establishing clear, dual-track criteria for safety and effectiveness, the CSEDB could become a foundational tool in building public trust and regulatory confidence in medical AI. As the technology advances, such rigorous, clinician-driven benchmarks will be essential to realizing AI’s full potential in improving patient outcomes.