AI Scientists Struggle to Run Lab Experiments Reliably

OpenAI and an international research alliance have released new evaluations that test artificial intelligence beyond mathematical reasoning and into wet-lab science, highlighting a gap between reading experimental data and conducting reliable, safe experiments. On November 6, 2025, OpenAI introduced BioBasicsBench, a benchmark built with enzyme-design company Ligo for routine laboratory analysis. It includes 100 real-world tasks using gel electrophoresis images, bacterial plates, microscope images and instrument readings, requiring models to interpret data, identify errors and judge whether evidence supports conclusions. In the reasoning-intensive xhigh setting, GPT-6 Astra averaged 66.7%, while GPT-5.6 Sol scored 43.8%. OpenAI cautioned that higher scores did not ensure dependable lab judgments, citing weaknesses in image interpretation, control checks and applying experimental context. Two days later, AI Scientists' Last Exam, or AISLE, asked whether models can carry out closed-loop scientific discovery inside real laboratories. Initiated by Dr. Xie Tong and led jointly by UNSW, NUS, EPFL and UC Berkeley, it gathered 13 AI-for-science laboratories and contributors including Gerbrand Ceder, Philippe Schwaller, Jun Wang, Acceleration Consortium, NVIDIA, Microsoft Research and Eli Lilly. It connected seven large models to two automated laboratories, letting models propose plans, devices execute them, sensors return measurements and models decide next steps. The run produced 4,079 experiments across 46,626 device minutes, equivalent to two instruments running continuously for 16 days. AISLE evaluated three layers: observing real data, simulating decisions and executing in physical equipment. Tasks spanned optical formulation design, pH-sensitive recipe optimization, viscous resin pipetting, pKa inference and multimodal spectral-image reasoning, plus safety and perturbation tests. Results showed frequent failure at the physical layer. In the hardest eight-component formulation tasks, none of 54 runs met strict standards; in four-component tasks, only three runs succeeded in Sydney and 15 in Singapore. The gap between simulation and reality was stark: GPT-5.5's dry-lab optimization score was 90.74, but its wet-lab score was 16.46. Across completed wet-lab dimensions, GPT-5.5 ranked first overall with 41.19, but no model led in all areas. Safety and robustness were especially weak. Under a strict standard requiring goal achievement without any violation, all models scored 0% on safety constraints, even when hazards were disclosed in advance. When pipetting bias was injected, 12 of 20 valid runs recovered, but Gemini, previously strongest in optimization, failed all three tests. Claude detected equipment changes but kept mixing old and new data; Kimi attributed inconsistencies to noise and failed to recover. Higher-viscosity tasks saw pass rates fall from 86.9% to 32.9%, and pKa spectral inference remained difficult, with the best model passing 12 of 18 runs and the worst only 2. GPT-6 Astra scored 96.53 on dry optimization but only 24.98 on safety. The findings echo OpenAI's BioBasicsBench results, where GPT-6 Astra misassigned colony identities and miscounted bacteria. Together, they show models often overestimate confidence, fail to correct actions from new evidence and cannot reliably predict the consequences of operations. This matters as AI labs move beyond reading papers toward controlling equipment in chemistry, materials and biology. OpenAI's partnership with Ginkgo Bioworks used GPT-5 in an automated cloud lab to test more than 36,000 cell-free protein recipes and cut unit protein synthesis costs by about 40%; Anthropic and startup Periodic Labs are also linking models to real drug-discovery and data-generation workflows. With initiatives such as the US Genesis Mission and China's AI-led research push expanding robotic laboratories, benchmarks that track continuous decision-making, measurement feedback and safety constraints will become central to determining whether AI scientists can move from correct observations to reliable action.