MIT researchers are sounding the alarm on the risk of patient data memorization in artificial intelligence models trained on electronic health records (EHRs), even when those records are de-identified. A new study presented at the 2025 Conference on Neural Information Processing Systems (NeurIPS) highlights how foundation models—large AI systems trained on vast amounts of medical data—can inadvertently memorize and later expose sensitive patient information, undermining the core principle of medical confidentiality. The Hippocratic Oath’s commitment to patient privacy remains a cornerstone of medical ethics, but it faces growing challenges in the era of AI. As health data becomes increasingly digitized and shared for model training, the risk of privacy breaches rises. The study, led by Sana Tonekaboni, a postdoctoral researcher at the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard, and co-authored by MIT Associate Professor Marzyeh Ghassemi of the Abdul Latif Jameel Clinic for Machine Learning in Health, investigates how AI models can leak private information through targeted prompts. While foundation models are designed to generalize knowledge from large datasets to make accurate predictions, they can also memorize specific patient records. This memorization occurs when a model retrieves and outputs information from a single training instance rather than synthesizing patterns across many records. Such data leakage poses a serious threat to patient privacy, especially when attackers use precise prompts to extract sensitive details. To assess the real-world risk, the research team developed a series of practical tests to evaluate how easily an attacker could recover private information. They found that the more details an attacker already possesses—such as lab test dates and values—the higher the chance of successful data extraction. However, the researchers stress that if an attacker already has access to such granular data, the need to exploit an AI model diminishes. The real danger lies in cases where minimal information is required to trigger a leak. The study also distinguishes between different types of data leakage. Revealing basic demographics like age or gender is less harmful than exposing highly sensitive conditions such as HIV status or substance abuse history. Patients with rare or unique medical conditions are particularly vulnerable, as their records can be re-identified more easily even after de-identification. The team emphasizes that privacy evaluation must be context-specific to healthcare. They advocate for standardized testing protocols that measure both the likelihood and severity of leaks, ensuring that models are safe before deployment. Future work will involve collaboration with clinicians, privacy experts, and legal scholars to build a more comprehensive framework for responsible AI in medicine. The research is supported by the Eric and Wendy Schmidt Center, the Knut and Alice Wallenberg Foundation, the U.S. National Science Foundation, the Gordon and Betty Moore Foundation, Google Research, and the AI2050 Program at Schmidt Sciences. Additional resources were provided by the Province of Ontario, the Government of Canada through CIFAR, and sponsors of the Vector Institute.