MIT Develops Millimeter-Wave Technology for Precise 3D Reconstruction of Objects in Boxes

Microsoft has lent its support to MIT researchers who have developed a groundbreaking imaging technology called mmNorm. This technology utilizes millimeter waves (mmWave) to create precise 3D reconstructions of objects hidden within plastic containers or behind walls. Millimeter waves, which occupy the 30 to 300 GHz frequency range with wavelengths between 1 to 10 millimeters, combine characteristics of microwaves and far-infrared radiation. These waves can penetrate common obstacles, enabling detailed reconstructions of objects not visible to the naked eye. The applications of mmNorm are diverse and promising. Sorting robots could use it to locate and manipulate objects in cluttered environments or sealed boxes. AR devices could visualize hidden items, enhancing user experiences in augmented reality settings. Smart home devices could recognize gestures even when users are out of direct line of sight, allowing for contactless commands. Additionally, radar systems using millimeter wave signals could detect aircraft obscured by clouds. However, existing methods for millimeter wave reconstruction have limitations, particularly in precision and efficiency when dealing with smaller objects like household items. Recently, MIT researchers addressed these issues by developing mmNorm, which significantly improves the accuracy of 3D reconstructions. The technique was evaluated over 110 real-world experiments involving more than 60 different everyday objects and compared against state-of-the-art benchmarks, achieving a 96% reconstruction accuracy. This is a marked improvement over the 78% accuracy of previous methods. A key innovation in mmNorm is its ability to interpret mirror-like reflections. When millimeter waves hit a surface, they often reflect like a mirror, only returning to the receiver if the surface is directly aligned with the antenna. If the surface is oriented differently, the reflected waves may miss the receiver. By analyzing the angles, time delays, and subtle variations in the reflected signals, mmNorm can infer the direction each point on an object's surface faces, estimating what’s known as the "surface normal" vector. Accumulating these vectors allows the system to reconstruct the object's 3D shape with high precision. To test mmNorm, the MIT team created a prototype by mounting a radar array on a robotic arm. The arm moved around hidden objects, continuously collecting measurement data. The system then compared signal strengths from different positions to estimate surface curvatures. For instance, the antenna receives the strongest reflection from surfaces directly facing it, while surfaces at other angles produce weaker signals. Each antenna "votes" on the likely direction of the surface normal based on the strength of the received signal. Some antennas have higher voting weights, and the system integrates all these votes to determine the most probable surface orientation. Laura Dodds, an MIT research assistant and lead author of the study, explained, "Certain antennas have very high voting weights, while others have low ones. We aggregate all these votes to achieve a consensus on the surface normals." This process generates a vast number of potential surfaces, and to pinpoint the correct one, the researchers employed techniques from computer graphics to construct a 3D function that best represents the received signals, ultimately creating a highly accurate 3D model. During their experiments, the team tested mmNorm on a variety of complex-shaped objects, including a mug with a handle, achieving a 96% accuracy rate. The method also excelled in reconstructing multiple objects within the same container, such as a set of silverware, and performed well across various materials, including wood, metal, plastic, rubber, and glass, though it cannot image through metals or excessively thick walls. The enhanced precision of mmNorm opens up numerous new possibilities. Robots could accurately identify and grasp specific tools in a toolbox by determining the exact shape and position of handles. AR devices combined with mmNorm could provide factory workers with realistic images of hidden objects. In security and military applications, the technology could improve the detection and reconstruction of concealed items. Looking forward, the researchers plan to refine mmNorm further. They aim to enhance the system's performance with low-reflection objects, increase its ability to penetrate thicker barriers, and continue improving resolution. Laura Dodds highlighted the broader implications of their work: "This research completely shifts our paradigm in understanding millimeter wave signals and the 3D reconstruction process. We anticipate that these breakthrough insights will have a wider impact." The study was supported by the National Science Foundation, the MIT Media Lab, and Microsoft, underscoring the collaborative effort behind this technological advancement.