Have you ever thought of turning your thoughts into words without moving your lips? A group of Chinese researchers has decided to make this "science fiction" a reality with a system to produce speech - or vocal synthesis - by interpreting the brain's neural activity.
Thoughts and words
Il South China Morning Post back that researchers have created a “mind-reading machine,” capable of turning human thoughts into spoken Mandarin.
The technique used iselectrocorticography (ECoG), which measures direct signals from the brain. And how do you get these signals? Through electrodes implanted in the brain during surgery.
The thousand faces of “tonal” language
Here comes the beauty. Tonal languages such as Mandarin, Vietnamese, Punjabi, and others use tone and pronunciation synergistically to communicate meaning. A word can change completely based on tones, as well as the usual consonant and vowel changes.
In their paper, the researchers explain that they have proposed a “divide-and-conquer” framework, hypothesizing that tone and basic syllable can be decoded separately from neural activity and then tonal speech can be synthesized by recombining them.
From the operating table to vocal synthesis: the tests
Five volunteers, who underwent brain surgery to remove a tumor, were the testing ground. During the surgery, with the volunteers awake, they were asked to pronounce a series of specific syllables (“ma” and “mi” in four different tones) while neural activity was recorded.
From these recordings, the team was able to refine the algorithms that interpret neural activity and could synchronize the ECoG recordings with the audio recordings.
A future of words… think!
It is clear that this research opens the way to a world of possibilities. These findings not only offer methods for treating anarthria (the total loss of speech), but could also increase the communication efficiency of brain-computer interfaces.
The research was published in the journal Science Advances, and I link it here.
