Watch all the Transform 2020 sessions on-demand here.
Seeing around the corners of walls may sound like the stuff of comic books and superhero movies, but a team at Boston University, Draper, and the Massachusetts Institute of Technology have developed an artificially intelligent (AI) system that does just that.
At the IEEE International Conference on Computational Photography (ICCP) in Tokyo, Japan, researchers presented a computational photography technique that isolates shadows cast by hidden objects and uses algorithms to reconstruct them.
“Imaging the scene behind a barrier can provide [a] tactical advantage in many real-life scenarios, for instance, autonomous vehicle navigation, and search and rescue,” Sheila Werth, the team’s lead investigator, said in a statement.
In a poster titled “Occluder-Aided Computational Periscopy,” the team describes the scenario: light from the hidden scene is cast onto the floor around the corner of the wall, forming a partial shadow, or penumbra — a subtle gradient of light that encircles the corner. (Penumbras use only 0.1 percent of the light that’s reflected off the ground, and often can’t be seen with the naked eye.) Using a combination of algorithms and geometry, the software separates the penumbra from the floor pattern and generates a model of the scene.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
It’s not the first system of its kind, strictly speaking.
In January 2017, researchers at Xi’an Jiaotong University in China developed a single-pixel camera that photographs behind-the-corner objects by recording the intensity of light scattered off a wall.
More recently, in October 2017, scientists at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) created an AI system that also uses penumbras to detect if there’s a person or object around the corner of a wall, and even estimate its trajectory and speed if it’s in motion.
But both techniques have fairly severe limitations. The Jiaotong researchers’ camera needs to capture light at a minimum of about 50,000 times before it can reconstruct a scene, and MIT’s algorithm can only detect objects from video recordings. Werth’s system, by contrast, works with a single ordinary photograph.
The team, which was co-advised by MIT professor William Freeman and Boston University professor Vivek Goyal, Draper researcher Chris Yu, and BU’s Yanting Ma, John Murray-Bruce, and Charles Saunders, was awarded “best poster” at ICCP 2018.
