Scientists from the Charles III University of Madrid (UC3M) have published a paper with the results of a research on the integration between robots and people with disabilities.
Beyond the interesting data, a part of the study stands out, which involved the planning of TEO, a humanoid robot, which has "learned" to communicate in sign language.
TEO (Task Environment Operator) is the result of a previous research carried out by the Robotics Lab, and has served as a "student" for this study.
To acquire this ability, a robot must be able to combine different areas: AI, neural networks, artificial vision, and express them through appropriately engineered robotic hands.
“One of the major new developments in this research is the union of two broad areas of robotics. Complex systems (hands or robotic arms) and communication / social interaction skills ". He explains it Juan Víctores, one of the Robotics Lab researchers in the Systems Engineering and Automation Department at UC3M.
In fact, the first activity carried out as part of this research was to instruct TEO. Through a simulation, the robot received indications on the specific position of each phalanx to express each element of the sign language.
The attempt to make the robot reproduce positions and movements very similar to those performed by a human being can already be said to be largely successful.
“The goal is to look natural. Various types of neural networks have been used to achieve this adaptation. The mix of gestures and breadth that can be understood by the people who communicate with this language ", clarifies the research.
In the last test phase, the scientists verified the functioning of TEO by making it interact with real users. "Those who came into contact with the robot gave 80% positive feedback, a more than satisfactory answer", says another of the researchers, Jennifer J. Gago.
It is true that the conversation developed by TEO is less than basic. The robot has dedicated itself to reproducing one by one the letters of the sign language alphabet and a limited dictionary of words related to familiar objects. He did it well, but it obviously must be considered a starting point. The challenge of this system is now to synthesize more complex gestures, entire sentences, coding criteria.
What TEO can do
When the process is complete, TEO will play a very important role in the relationship with the more than 13000 people in Spain who use sign language, and with others in the world.
TEO and sign language will be able to replace Google Home or Alexa as a virtual assistant that communicates directly with gestures. Not only that: considering the fact that TEO has the ability to iron (I'm not kidding, it's one of the “learned” skills), fold clothes, serve food and interact with users who use other languages.
TEO can become a sign language interpreter to be hired in every public building to allow everyone to access the services of a municipality.
The really important thing about these studies is their destination in society: and technology thus becomes a tool of true inclusion.
More info: Jennifer J. Gago et al. Sign Language Representation by TEO Humanoid Robot
