Humans still outperform AI in reading social dynamics and context in moving scenes, a crucial skill for technologies like self-driving cars and assistive robots. This finding comes from a study led by researchers at Johns Hopkins University, highlighting a significant gap in AI capabilities when it comes to understanding human interactions. Leyla Isik, an assistant professor of cognitive science at Johns Hopkins, explained the importance of AI comprehending social cues. "For a self-driving car, recognizing the intentions, goals, and actions of human drivers and pedestrians is essential. It needs to predict if a pedestrian is about to start walking or if a pair of people are in conversation rather than preparing to cross the street. Any AI system that interacts with humans must be able to interpret what people are doing," she said. Kathy Garcia, a doctoral student who co-authored the study, will present the findings at the International Conference on Learning Representations on April 24. The research involved comparing human and AI judgments of social interactions in short video clips. Human participants watched three-second videos depicting various scenarios, including people interacting, performing side-by-side activities, or engaging in independent tasks. They then rated the importance of certain features for understanding these social interactions on a scale from one to five. The team also evaluated over 350 AI models—ranging from language, video, and image models—to predict human ratings and brain responses to the videos. For language models, the AIs assessed brief, human-written captions of the scenes. Despite the varying sizes and training data, the AI models generally failed to align with human perceptions. Video models struggled to accurately describe the actions in the clips, and even image models provided unreliable predictions about whether individuals were communicating. While language models were somewhat better at predicting human behavior, video models showed more promise in predicting neural activity in the brain. These results starkly contrast with AI's proficiency in analyzing static images. Isik noted, "Recognizing objects and faces in still images was a major milestone in AI, but real life is dynamic. AI needs to understand the unfolding story in a scene, including the relationships, context, and dynamics of social interactions. Our research indicates that AI has a significant blind spot in this area." The researchers attribute this discrepancy to differences in the neural networks that power AI and the human brain's specialized areas for processing dynamic social scenes. "The neural infrastructure that processes static images is distinct from the part that handles dynamic interactions," said Garcia. "There are many nuances, but the key point is that AI models are missing something fundamental in how they process these complex scenes." This insight underscores the need for more advanced AI development that focuses on capturing the subtleties of dynamic social environments. Current AI systems excel at identifying static elements but fall short in understanding the ever-changing, intricate nature of human activities and interactions. The findings suggest that designing AI with a more comprehensive approach to scene processing, akin to how the human brain operates, could bridge this gap and improve the safety and efficiency of AI-driven technologies.