Scientists from the Charles III University of Madrid (UC3M) have published a paper with the results of research on the integration between robots and people with disabilities.
Beyond the interesting data, a part of the study stands out, which involved the planning of TEO, a humanoid robot, who has “learned” to communicate in sign language.
TEO (Task Environment Operator) is the result of previous research by the Robotics Lab, and acted as a "student" for this study.
To acquire this ability, a robot must be able to combine different areas: AI, neural networks, artificial vision, and express them through appropriately engineered robotic hands.
“One of the main new developments of this research is the merging of two broad areas of robotics. Complex systems (hands or robotic arms) and communication/social interaction skills”. He explains it Juan Victores, one of the Robotics Lab researchers in the Systems Engineering and Automation Department at UC3M.
In fact, the first activity carried out as part of this research was to train TEO. Through a simulation, the robot received indications on the specific position of each phalanx to express each element of the sign language.
The attempt to make the robot reproduce positions and movements very similar to those performed by a human being can already be said to be largely successful.
“The goal is to look natural. Various types of neural networks have been used to obtain this adaptation. The mix of gestures and breadth understandable by people who communicate with this language", clarifies the research.
In the last test phase, scientists verified how TEO works by making it interact with real users. “Those who came into contact with the robot gave 80% positive feedback, a more than satisfactory response”, says another of the researchers, Jennifer J. Gago.
It is true that the conversation developed by TEO is less than basic. The robot dedicated itself to reproducing one by one the letters of the sign language alphabet and a limited dictionary of words linked to familiar objects. He did it well, but it obviously must be considered a starting point. The challenge of this system is now to synthesize more complex gestures, entire sentences, coding criteria.
What TEO can do
When the process is complete, TEO will play a very important role in the relationship with the more than 13000 people in Spain who use sign language, and with others in the world.
TEO and sign language will be able to replace Google Home or Alexa as a virtual assistant that communicates directly with gestures. Not only that: considering the fact that TEO has the ability to iron (I'm not joking, it's one of the "learned" skills), fold clothes, serve food and interact with users who use other languages.
TEO can become a sign language interpreter to be hired in every public building to allow everyone to access the services of a municipality.
The really important thing about these studies is their destination in society: and technology thus becomes a tool of true inclusion.
More info: Jennifer J. Gago et al. Sign Language Representation by TEO Humanoid Robot
