An implanted electrode picks up electrical signals, which are then translated to vocal movements and then speech sounds. Researchers use electrodes similar to this one to decode brain activity. (Photo from UCSF)
There are many diseases that can leas to loss of speaking, such as motor neurone disease, Parkinson’s, and multiple sclerosis. There are methods available that allow people to have some form of communication, but scientists at the University of California have developed a ground breaking method to help people find their voice once again. The team has created a brain-machine interface that can decode and translate brain activity directly into speech.
“For the first time … we can generate entire spoken sentences based on an individual’s brain activity,” said Edward Chang, a professor of neurological surgery at the University of California San Francisco (UCSF) and the senior author of the work. “This is an exhilarating proof of principle that, with technology that is already within reach, we should be able to build a device that is clinically viable in patients with speech loss.”
So how does it work? The decoding system is a two-step process. The first involves implanting an electrode in the brain that will pick up electrical signals that control the movement of the lips, tongue, voice box, and jaw. These signals in the brain are translated to vocal moments and then translated to speech sounds.
For testing, scientists collected data from volunteers who were about to undergo neurosurgery for epilepsy. They were implanted with the electrodes and then asked to read several hundred sentences out loud, while the scientists recorded the activity from an area in the brain known to have a hand in speech production. This data was compared to previously collected data by old research that tracked how vocal movements and speech sounds are linked. Using this they were able to determine the patients’ vocal movements.
Both sets of data were put together by training a machine learning algorithm to match the patterns of electrical activity in the brain with the vocal movements it would produce, like pressing the lips together. This resulted in what they call a “virtual vocal tract” that can be controlled by the brain to create a synthetic version of a person’s voice.
The final result is promising, but not perfect. Listening to an audio sample, it’s not crystal clear, but you can determine what they’re saying. When scientists shared the samples with various people and asked for feedback, 70% of listeners were able to make out what was being said albeit with a strong accent.
While it has the potential to restore someone’s speaking abilities, many question the ethics behind reading someone’s thoughts. Professor Chang reassures that the system cannot read private thoughts because it’s too hard. "We and others actually have tried to look at whether it's actually possible to decode just thoughts alone. And it turns out, it's a very, very difficult and challenging problem. That's only one reason why, of many, we really focus on what people are actually trying to say." Still, it’s a great step forward in helping those who have lost their ability to speak.
Have a story tip? Message me at: cabe(at)element14(dot)com