A global scoping review led by researchers at the University of Adelaide has uncovered significant inconsistencies in how artificial intelligence tools are tested in healthcare settings during a phase known as silent trials. Published in Nature Health, the study highlights the absence of standardized guidelines for this early testing stage, which occurs when AI systems are deployed in real-world clinical environments without alerting healthcare providers or patients that they are being used. The review examined numerous silent trials—where AI tools operate in the background, assisting clinicians with diagnostics, treatment recommendations, or administrative tasks—revealing wide variations in trial design, data collection methods, and the metrics used to evaluate performance. Some studies measured accuracy and clinical outcomes, while others focused on workflow efficiency or user satisfaction, with little consistency across projects. The lack of uniform standards raises concerns about the reliability, transparency, and ethical oversight of AI deployment in healthcare. Without clear protocols, it becomes difficult to compare results across different trials, assess risks, or ensure patient safety and informed consent. In many cases, clinicians were unaware they were interacting with an AI system, and patients were not informed at all, which challenges ethical principles of transparency and autonomy. The researchers emphasize that while silent trials can provide valuable insights into how AI tools perform in real-world conditions, the absence of regulatory frameworks and reporting standards undermines their scientific credibility and public trust. They call for the development of international guidelines to standardize trial methodologies, improve data sharing, and ensure ethical accountability. The study urges healthcare institutions, regulators, and AI developers to collaborate on establishing transparent, reproducible, and ethically sound processes for early-stage AI testing. Without such measures, the rapid integration of AI into clinical practice risks compromising patient safety and the integrity of medical research.