The researchers of the Carnegie Mellon University (CMU) have developed a camera system that can detect sound vibrations with a level of accuracy that allows you to recreate audio without inferences or a microphone. The company? Work of the team of Robotics Institute (RI) of the CMU School of Computer Science to build the system, consisting of two cameras and a laser.
The camera system can detect "high-speed, low-amplitude surface vibrations" that the human eye cannot see, the university said in a press release.
Low cost cameras, very high performance
The system is equipped with normal cameras instead of the high-speed ones used in previous searches, which will lead to a significant cost reduction. “We have made the optical microphone much more practical and usable,” he said Srinivasa Narasimhan, Professor of Robotics and Head of the Lighting and Imaging Laboratory. “We have improved quality while reducing costs”.
An algorithm compares the patterns of spots captured by a roller shutter and a global shutter. Use the differences between the models to calculate vibrations and recreate the audio. A speckled pattern (which in this case is created by the laser) refers to the behavior of coherent light in space after it is reflected off a rough surface. This behavior changes when the surface vibrates. The roller shutter quickly scans an image from one end to the other, while a roller shutter captures an entire image at the same time.
"This system pushes the boundaries of what can be done with computer vision," assistant professor Matthew O'Toole, co-author of a document on the system, he said. "This is a new mechanism for capturing high-speed, tiny vibrations and presents a new area of research."
Isolate the different sounds with a normal video shoot
The researchers say they were able to isolate the sound of the guitars that were being played at the same time. They claim the system was able to frame a bag of chips and only use its vibrations to reconstruct its sound. The audio was transmitted from a loudspeaker with a fidelity never achieved before.
There are many potential applications for this technology. The researchers suggest, for example, that the system could monitor vibrations from machines in a factory to look for signs of problems. Sound engineers might also isolate the sound from an instrument to enhance the mix. Essentially, it could help eliminate ambient noise from audio recordings.
