Study Finds Chatbots Provide Inconsistent Alcohol and Breast Cancer Data
A recent study presented at the 49th annual scientific meeting of the Research Society on Alcohol in San Antonio, Texas, and partially published in Public Health, reveals significant inconsistencies in how generative AI chatbots communicate the established link between alcohol consumption and breast cancer risk. Led by Allison Brandt Anbari, assistant professor at the University of Missouri’s Sinclair School of Nursing, the research evaluated 66 AI-generated responses across 22 publicly available chatbot platforms, including ChatGPT, Gemini, and Copilot. Researchers inputted three distinct prompts modeled after questions from authoritative health organizations: the World Health Organization, the Centers for Disease Control and Prevention, and the National Breast Cancer Foundation. Despite the causal relationship between alcohol and breast cancer being classified as a Group 1 carcinogen since 1988, the AI outputs varied substantially in both scientific accuracy and presentation. Many models incorrectly suggested that the underlying science remains complex or unresolved, contradicting established medical consensus. The analysis also found that readability scores consistently fell at the college level, far exceeding the National Institutes of Health recommendation of a sixth-grade reading level for public health materials. Crucially, the study demonstrated that minor variations in prompt phrasing directly influenced the structure, tone, and content of AI responses. Anbari emphasized that this discrepancy underscores a critical gap between health literacy, digital navigation skills, and prompt engineering proficiency. As consumers increasingly rely on conversational AI for medical guidance, these inconsistencies pose a tangible public health risk. Healthcare providers and policy stakeholders are urged to recognize the volatility of AI-generated health information and proactively address potential misinformation in clinical and educational settings. The findings call for standardized AI safety protocols and improved prompt transparency to ensure that public health messaging remains accurate, accessible, and consistent across all generative platforms.
