The researchers of the Carnegie Mellon University (CMU) have developed a camera system that can detect sound vibrations with a level of precision that allows audio to be recreated without inference or microphone. The company? Work of the team of Robotics Institute (RI) of the CMU School of Computer Science to build the system, consisting of two cameras and a laser.
The camera system can detect “high-speed, low-amplitude surface vibrations” that the human eye cannot see, the university said in a press release.
Low cost cameras, very high performance
The system is equipped with regular cameras rather than the high-speed ones used in previous research, which will lead to a significant reduction in costs. “We've made the optical microphone much more practical and usable,” he said Srinivasa Narasimhan, professor of Robotics and head of the Illumination and Imaging Laboratory. “We have improved quality while reducing costs.”
An algorithm compares the speckle patterns captured by a shutter and a global shutter. Uses the differences between the models to calculate vibrations and recreate audio. A speckled pattern (which in this case is created by the laser) refers to the behavior of coherent light in space after it has been reflected from a rough surface. This behavior changes when the surface vibrates. The rolling shutter quickly scans an image from one end to the other, while a rolling shutter captures an entire image at once.
“This system pushes the limits of what can be done with computer vision,” assistant professor Matthew O'Toole, co-author of a document about the system, he said. “This is a new mechanism for capturing high-speed, tiny vibrations and presents a new area of research.”
Isolate the different sounds with a normal video shoot
The researchers say they were able to isolate the audio of the guitars being played at the same time. They claim that the system was able to focus on a bag of chips and use only its vibrations to reconstruct its sound. The audio was transmitted from a speaker with a fidelity never achieved before.
There are many potential applications for this technology. The researchers suggest, for example, that the system could monitor vibrations from machines in a factory to look for signs of problems. Sound engineers might also isolate the sound from an instrument to enhance the mix. Essentially, it could help eliminate ambient noise from audio recordings.
