A computer lab working on simplifying human-computer interaction for people of all abilities has developed a digital lip reader with its own machine learning system that helps the user understand better.
LipType It's the professor's invention Ahmed Sabbir Arif and his laboratory. In short: read the lip movement. It reads it so well that it allows people to send text messages or emails on PCs and smartphones, and to have contactless interactions with public devices such as ATMs or other kiosks. without speaking aloud.
There are other lip-reading devices, but they are not widely used because they are slow and often faulty.
A leap forward
There are many errors in dictation, especially in noisy places. People with language problems also have problems making themselves understood, perhaps due to their accent. LipType works for anyone. People may need to send a private message while in a public space, or in a meeting, and with LipType they can just move their lips without making a sound.
Arif his students have added various filters with different algorithms artificial intelligence to allow lip reading in different lighting conditions. They then introduced an error corrector based on different language models and found that LipType was significantly faster than other existing devices.
The lip reading test
To accompany the software test, Arif's lab conducted a social study to see if people would use such a technology. They contacted students and people from the community, including people with disabilities, and conducted an online survey. The reactions were enthusiastic.
The software test results were also outstanding. “LipType performed 58% faster than other models,” the researcher says Laxmi Pandey. “The device has proven to be state-of-the-art in various real-world settings, with low light and noise. The success of LipType makes me believe that it can revolutionize our interaction with each other and with computing devices."
The social study and the LipType lip reader study have been accepted for publication and presentation at the major international conference on Human-Computer Interaction.
The possible applications of the LipType lip reader
There are several possible applications for this lip reading system. Obviously LipType can bring people with disabilities much closer to digital tools. In many cases it could be useful to the police to understand what happens in scenarios in which they cannot access by listening. Car interfaces can also be useful, so you can write messages even while listening to music. There is a design philosophy for everything: the limit is your imagination.
