“See” Around Corners With Smartphone Tech

In spy novels and superhero films, the ability to see through walls has always been a handy — not to mention, impressive — trick. And now, this tech could be available to people in real life, with smartphone cameras that can help detect moving objects even if they are hidden around corners, according to a new study.

This futuristic-sounding tech could one day help vehicles see around blind corners, the researchers said.

“We may eventually be able to use this idea to alert drivers to pedestrians or cars that are about to dart out from behind buildings into a driver’s path. Perhaps a few seconds of notice could save lives,” said study lead author Katie Bouman, an imaging scientist at the Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory.

“Search and rescue, or helping to understand what is going on behind a wall in a hostage situation, are also potential applications,” Bouman added.

Researchers have taken many different approaches in trying to make the “superpower” of seeing around corners a reality. For example, in 2015, researchers showed they could use lasers to see objects around corners by firing light pulses at surfaces near the items. Those surfaces could act like mirrors, scattering the laser pulses onto any hidden objects. By analyzing the light that was reflected off the objects and other surfaces back onto the scanners, researchers could reconstruct the shapes of the hidden items.

Although most strategies for seeing around corners “are really great ideas,” they also “usually require complex modeling [or] specialized hardware, or are computationally expensive,” Bouman told Live Science. The 2015 study’s technique, for example, required both extremely fast lasers and extraordinarily sensitive cameras.

But Bouman and her colleagues’ method for seeing around corners simply uses a smartphone camera.

“We use light naturally in the scene and do not have to introduce our own light to probe the hidden scene,” Bouman said. “This allows us to use common consumer cameras and not specialized equipment to see around corners.”

The new system, known as CornerCameras, analyzes light that is reflected off objects hidden around corners and that falls on the ground within the line of sight of the camera. This light is called the “penumbra.”

The system analyzes this penumbra over several seconds, stitching together dozens of distinct images, according to the study. This data helps the system measure the speed and trajectory of objects around corners in real time. (It does not see any identifying details about those objects — just the fact that they are moving.)

“I think the biggest surprise was that the system worked well in situations that I would not have expected,” Bouman said. “For instance, once, during filming, it started raining. This caused big raindrops to start appearing on the ground, changing the color of the concrete floor.”

Because CornerCameras is trying to analyze light signals that are just 0.1 percent of the total brightness of the ground, “I thought these raindrops would wipe out any signal we had,” Bouman said. However, CornerCameras analyzes the data of a scene across dozens of images, so “the effect of the raindrops was essentially averaged out.”

One current limitation of CornerCameras is that it requires a stationary camera that’s held very steady. “In many situations, such as in a collision-avoidance system on a car, you do not have the luxury of a stationary camera,” Bouman said. The researchers are now focused on getting the system to work first on a moving wheelchair and eventually on a moving car, she said.

Future research will also aim to make CornerCameras work in a variety of lighting situations, or in changing lighting conditions, such as when clouds overhead constantly move in front of the sun. “Getting the system to work in these scenarios would open up the possibility of it being able to be used by a person with a handheld smartphone,” Bouman said.

Bouman and her colleagues will detail their findings on Oct. 25 at the International Conference on Computer Vision in Venice, Italy.