Today's sensing systems require complex components and elaborate algorithms that analyze frame by frame, making them inefficient and energy-consuming.
A new neuromorphic technology inspired by the human visual system integrates sensor, memory and processing in a single device. And it is able not only to detect the movement of objects, but to predict their trajectories.
The sensor that sees and predicts from just one frame
The neuromorphic sensor developed by Aalto University in Finland is based on a group of photomemristors, electrical devices that produce electric current in response to light.
When the light goes out, the current does not stop immediately, but gradually decreases. In practice, photomemristors "remember" whether they have been exposed to light recently, allowing the sensor to record not only instantaneous information, but also to store previous moments.
Hongwei Tan is the researcher who led the study published in Nature Communications (I link it here). The unique feature of this technology, he explains, is the ability to integrate a series of optical images into a single frame. The result is a compact and efficient detection unit.
Image: Hongwei Tan / Aalto University
Studies
To demonstrate the validity of the technology, the researchers carried out two very interesting experiments.
In the first, they used videos that show the letters of a word one at a time. The neuromorphic sensor was able to use the information hidden in the last frame to deduce which letters preceded it and predict with almost 100% accuracy what the word was.
In the second, the team showed the sensor video of a simulated person moving at three different speeds. Not only was the system able to recognize motion by analyzing a single frame, but it also correctly predicted subsequent frames.
The future
What is this sensor for? Recognizing and accurately predicting the future motion and location of objects is crucial for autonomous driving technology and intelligent transportation.
To “make decisions,” autonomous vehicles need highly accurate predictions about how cars, bicycles, pedestrians and other objects move around them.
For this reason, even without sensors, for this technology it is easy to predict the next frame, indeed: the successful frame.
