Scientists from the prestigious Massachusetts Institute of Technology (MIT) have developed a revolutionary imaging technology that has the potential to fundamentally change the way we perceive the physical world around us. Their innovative system, called mmNorm, enables the precise three-dimensional reconstruction of objects hidden behind obstacles such as cardboard boxes, plastic containers, or even interior walls. This ability, reminiscent of X-ray vision from science fiction, opens the door to incredible applications in robotics, quality control, security, and augmented reality.

A revolution in seeing the invisible

Imagine a robot in a warehouse that can "peek" inside a sealed package and determine if the fragile contents, like a ceramic mug, were damaged during transport before even opening it. Precisely such a scenario becomes possible thanks to a new approach that uses millimeter waves (mmWave). This is the same type of signal used in some advanced Wi-Fi systems, whose key characteristic is the ability to penetrate common materials that are opaque to the human eye and optical cameras. The research team from MIT, located in the city of Cambridge, Massachusetts, has developed a method that not only detects hidden objects but also reconstructs their complete 3D shape with exceptional precision.

The mmNorm system collects reflected millimeter waves and forwards them to a sophisticated algorithm that estimates the exact surface geometry of the hidden object from this data. In tests conducted on a wide range of everyday objects with complex, curved shapes – from cutlery to an electric drill – the new technique achieved an accuracy of 96 percent in reconstruction. For comparison, the most advanced existing methods achieved only 78 percent accuracy under the same conditions, indicating a huge leap in performance.

Overcoming the limitations of traditional radar

Classical radar techniques, which have been used for decades to detect distant or obscured objects like airplanes in clouds, rely on a method known as back-projection. Although effective for large objects, the resolution of such imaging is too coarse to discern the fine details of smaller items, which is crucial for applications in modern robotics. Fadel Adib, an associate professor in the Department of Electrical Engineering and Computer Science and head of the Signal Kinetics group in the MIT Media Lab, points out that previous methods had hit a wall. "It was necessary to devise a completely different way of using these signals to unlock new types of applications," states Adib, who led the research along with Laura Dodds, Tara Boroushaki, and Kaicheng Zhou.

The MIT team realized that existing techniques ignore a key property known as specularity. When millimeter waves hit a surface, it behaves almost like a mirror, creating mirror-like (specular) reflections. If a part of the surface is aimed towards the radar antenna, the signal will bounce back and be received. However, if the surface is turned, the reflection will travel in another direction and be lost to the sensor. "Relying on specularity, our idea is to estimate not only the location of the reflection in the environment but also the orientation of the surface at that point," explains Laura Dodds, the lead author of the study.

How does mmNorm work?

At the heart of the mmNorm technology lies the ability to estimate the so-called "surface normal," which is essentially a vector that indicates the direction of the surface at a specific point in space. By combining the estimates of normals from all the points from which the signal was reflected, the system uses an advanced mathematical formulation to reconstruct the entire 3D shape of the object.

The prototype system consists of a radar attached to a robotic arm. As the arm moves around the space where the hidden object is located, the radar continuously takes measurements. The system compares the strength of the signals received at different positions to estimate the curvature of the surface. For example, the antenna will receive the strongest reflection from the part of the surface facing it directly, and weaker signals from those parts that are at an angle. Since the radar has multiple antennas, each of them "votes" on the direction of the surface normal based on the strength of the signal it received. "Some antennas might have a very strong vote, some a very weak one, and we combine all the votes to get a single surface normal that all antenna locations agree on," adds Dodds. This process results in a point cloud with associated normals, from which a final, detailed 3D reconstruction is generated using techniques borrowed from computer graphics.

A wide range of applications and future potential

Besides achieving a significantly lower reconstruction error (about 40 percent less than existing approaches), mmNorm can also accurately distinguish between multiple objects hidden together, such as a fork, knife, and spoon in the same box. The technology has proven effective on objects made from various materials, including wood, metal, plastic, rubber, and glass. The only current limitation is objects hidden behind thick metal barriers or very thick walls.

The potential applications are vast and varied:

• Industry and logistics: Robots equipped with this technology can distinguish tools in a box, precisely determine the shape and position of a hammer's handle, and pick it up safely. In warehouses, automated quality control can be performed without opening packages.


• Augmented reality (AR): A factory worker or a construction worker could use AR glasses that would show them real-time, faithful images of completely hidden objects, such as pipes or electrical wiring inside a wall, before they start drilling.


• Security and defense: The technology could be integrated into existing security applications, such as airport scanners, allowing for more precise reconstructions of hidden objects. It also has potential in military reconnaissance.


• Home assistance: In the future, it could be used in systems to assist the elderly, helping to locate frequently lost items.

Tara Boroushaki, one of the researchers, emphasizes that the qualitative results speak for themselves and that "the amount of improvement makes it easier to develop applications that use these high-resolution 3D reconstructions for new tasks." The team plans to continue working on improving the resolution, increasing efficiency with less reflective objects, and enabling imaging through even thicker obstacles. "This work truly represents a paradigm shift in how we think about these signals and the 3D reconstruction process," concludes Dodds, hinting at an exciting future where the invisible world will become visible.

Source: Massachusetts Institute of Technology