Have you ever thought of turning your thoughts into words without moving your lips? A group of Chinese researchers has decided to make this "science fiction" a reality with a system for producing speech - or speech synthesis - by interpreting the neural activity of the brain.
Thoughts and words
Il South China Morning Post back that researchers have created a "mind-reading machine," capable of turning human thoughts into spoken Mandarin.
The technique used is theelectrocorticography (ECoG), which measures direct signals from the brain. And how do you get these signals? Through electrodes implanted in the brain during surgery.
The thousand faces of "tonal" language
Here comes the beauty. Tonal languages such as Mandarin, Vietnamese, Punjabi, and others use tone and pronunciation synergistically to communicate meaning. A word can change completely based on tones, as well as the usual consonant and vowel changes.
In their paper, the researchers explain that they have proposed a "divide-and-conquer" framework, assuming that tone and base syllable can be decoded separately from neural activity and then tonal speech can be synthesized by recombining them.
From the operating table to vocal synthesis: the tests
Five volunteers, who underwent brain surgery to remove a tumor, were the testing grounds. During the intervention, with the volunteers awake, they were asked to pronounce a series of specific syllables ("ma" and "mi" in four different tones) while the neural activity was recorded.
From these recordings, the team was able to refine the algorithms that interpret neural activity and were able to synchronize the ECoG recordings with the audio recordings.
A future of words… think!
It is evident that this research opens the way to a world of possibilities. These discoveries not only offer methods for the treatment of anarthria (total loss of speech), but could also increase the communication efficiency of brain-computer interfaces.
The research was published in the journal Science Advances, and I link it here.
