A team from MIT has developed an innovative system called Themis AI, designed to perform model self-inspection and correction in just a few seconds. This system is compatible with various models, addressing the crucial issue of ensuring AI reliability. The journey of Themis AI began several years ago in a laboratory at MIT, where Professor Rus led a team focused on a fundamental question: how can machines recognize their own limitations? In 2018, Rus’s team received funding from Toyota to explore this issue, specifically aiming to improve the reliability of autonomous driving systems. “In safety-critical environments, understanding a model's reliability is extremely important,” Rus explained. Research results showed that combining specialized uncertainty estimation algorithms with advanced autonomous driving technologies could reduce collisions by 16 times, decrease computation time by 12 times, and boost recovery rates after near-collisions by 89%. Additionally, it could cut down on automatic driving requests by 93%. In another project, Rus and her colleagues Amini and their team developed a method to detect biases in facial recognition systems. By automatically reweighting training data, they successfully eliminated these biases. The process involves identifying underrepresented characteristics in the training data and generating new similar samples to achieve a balanced dataset. By 2021, the team had demonstrated that this approach could assist pharmaceutical companies in using AI models to predict drug properties more accurately. "Guiding drug discovery can save enormous costs, which makes this application particularly significant," Rus noted. The method reduced drug development costs by 75%, increased research speeds by 10 times, and decreased the amount of required training data by 60%. Around the same time, they founded Themis AI, which now collaborates with multiple industries, especially those building large language models. Using the Capsa platform, these models can analyze their outputs and report their confidence levels, thus helping to identify hidden unreliable results before taking action. One of the key advantages of Themis AI is its suitability for edge devices with limited computational power. Edge devices often have smaller models due to size constraints, which cannot match the precision of server-based larger models. However, with Themis technology, these devices can handle a multitude of tasks more effectively locally, only requesting server assistance when facing challenges. “Typically, the small models used in smartphones or embedded systems are less accurate than server versions, but our technology ensures both low latency and high efficiency without sacrificing quality. We envision a future where edge devices will perform the primary tasks, but when their output is questionable, any task can be seamlessly handed over to middleware servers for processing,” said Stewart Jamieson, the technology manager of Themis AI. Pharmaceutical companies can also leverage Capsa to optimize AI models for drug selection and clinical trial prediction. This versatile technology is set to revolutionize various sectors, enhancing both efficiency and accuracy while maintaining safety standards.