Bioengineers from the University of California, Los Angeles, designed a glove-like device that can translate sign language into hearable words in real time. Research was published in the journal Nature Electronics.
Jun Chen, assistant professor of bioengineering at the UCLA Samueli School of Engineering is the lead author of the research. “Our hope is that this technology will become a simple tool for translating sign language and communicating directly with those who don't know it without the need for an intermediary,” he said. “We also hope it can help more people learn sign language on their own.”
How translator gloves are made
The system consists of a couple of wearable devices. These are gloves with thin, stretchable sensors that run the length of each of the five fingers. These sensors, made with electrically conductive yarns, detect hand movements and finger positioning that indicate individual letters, numbers, words and phrases.
The device then transforms finger movements into electrical signals, which are sent to a wrist device the size of a coin. The card transmits these signals wirelessly to a smartphone which translates the sign language into words at the rate of about one word per second.
The researchers also added adhesive sensors to the testers' faces (between the eyebrows and sides of the mouth) to capture facial expressions that are part of sign language.
Translating sign language is easier
Previous wearable systems that offered translation from sign language were limited by bulky and heavy devices or were uncomfortable to wear, Chen said.
The device developed by the UCLA team is made from lightweight, inexpensive but long lasting and stretchable polymers. Electronic sensors are also very flexible and inexpensive.
In testing the device, the researchers worked with four deaf people who use sign language. The wearers repeated each hand gesture 15 times. A custom machine learning algorithm transformed these gestures into the letters, numbers, and words they represented. The system recognized 660 signs, including every letter of the alphabet and numbers from 0 to 9.
The improvement of this technology will allow to break down an important barrier. Its miniaturization will mean that one day instead of gloves there may be simple wearable rings capable of translating sign language. Or it will open up to virtual reality environments where there is immediate dialogue between everyone, regardless of language (verbal or non-verbal) known.
