A team at Carnegie Mellon University’s Human-Computer Interaction Institute is turning ordinary objects into intelligent, proactive assistants using AI and robotics. Their latest innovation features a stapler mounted on a small wheeled platform that can autonomously slide across a desk to meet a user’s hand just as they reach for it. Similar systems enable a knife to move out of the way before someone leans on the counter, or a mug to glide into position when someone reaches for a drink. This technology, developed by the Interactive Structures Lab led by Assistant Professor Alexandra Ion, combines computer vision, large language models (LLMs), and robotic mobility to give everyday items the ability to anticipate human needs. The system uses a ceiling-mounted camera to monitor the environment, capturing real-time video of people and objects. This visual data is translated into a text description of the scene, which an LLM then analyzes to infer the user’s likely goals and the best actions to support them. Once the system predicts what the user needs, it sends commands to the object’s robotic base, allowing it to move across a surface—like a desk or countertop—without disrupting the space. The process is designed to be unobtrusive, meaning users don’t need to issue commands. Instead, the object acts independently, based on its understanding of behavior and context. The research was presented at the 2025 ACM Symposium on User Interface Software and Technology in Busan, Korea. Ion emphasized that the goal is to create physical AI systems that blend seamlessly into daily life, offering help without drawing attention. “We want AI assistance in the physical world to be as natural and intuitive as it is in digital spaces,” she said. Ph.D. student Violet Han, who works closely with Ion, noted that enhancing familiar objects builds trust. “People already rely on things like staplers and mugs. By giving them intelligence, we can extend that trust into new capabilities.” The team envisions broader applications, such as a kitchen shelf that automatically extends when someone walks in with groceries, or a hospital bed that adjusts itself based on a patient’s movements. The long-term vision is to develop safe, reliable, and invisible AI that enhances everyday environments—homes, offices, factories, and healthcare settings—by making physical interactions smarter and more responsive. The research, titled “Towards Unobtrusive Physical AI: Augmenting Everyday Objects with Intelligence and Robotic Movement for Proactive Assistance,” is published in the proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology.