A man with a debilitating neurological disorder can “speak” again thanks to a computer that translates brain activity into text read aloud.
Scientists have even succeeded in imitating Casey Harrell’s voice using recordings from before his illness.
8
8
8
The father, whose brain had sensors implanted, said: “Not being able to communicate is so frustrating and demoralizing.”
“It’s like being trapped – often isolated and lonely.
“Something like this technology will help people get back into life and society.”
Heartwarming footage shows the 45-year-old with his family as he tries out the device for the first time.
“He started crying, I started crying, his family started crying,” said Dr. Sergey Stavisky, a neuroscientist who oversaw the trial.
“Everyone had to stop and collect themselves. It was a very special moment.”
Casey was diagnosed with amyotrophic lateral sclerosis (ALS) – a form of motor neuron disease (MND) – in 2020.
He noticed the first symptoms after the birth of his daughter.
At first, doctors suspected that he was suffering from muscle twitches that would go away on their own.
Casey was subsequently diagnosed with Lyme disease, which can resemble the early symptoms of ALS, before neurologists finally realized that his limp and spasms were due to the muscle-wasting disease.
His condition deteriorated noticeably and he is now dependent on a special wheelchair.
Casey can no longer stand or walk and no longer has control over the muscles he needs to speak.
However, scientists have developed a new brain-computer interface (BCI) that can interpret his brain signals when he tries to speak.
They are then converted into text, which is “spoken” out loud by the machine with 97 percent accuracy.
It makes people cry who haven’t heard me in a while
Casey Harrel
Dr. David Brandman, a neurosurgeon at UC Davis Health, said, “Our BCI technology has helped a paralyzed man communicate with friends, family and caregivers.
“Our work presents the most precise speech neuroprosthesis (the most accurate device) ever reported.”
The BCI even sounds just like Casey because the researchers used artificial intelligence (AI) to mimic his voice.
“Using video recordings taken before he developed ALS, we were able to digitally reconstruct his voice,” said Dr. Brandman.
“So now when he tries to speak, not only do words appear on his screen, but it also sounds like him.”
“It feels very much like me,” said Casey, who is still able to work full time.
“It makes people cry who haven’t heard me in a while.”
8
8
8
What is ALS?
ALS or amyotrophic lateral sclerosis is a progressive neurodegenerative disease that affects nerve cells in the brain and spinal cord.
It is the most common form of motor neuron disease (MND).
MND occurs when special nerve cells in the brain and spinal cord, called motor neurons, no longer function properly and die prematurely.
Motor neurons control muscle activities such as grasping, walking, speaking, swallowing and breathing.
As the disease progresses, it may become more difficult to perform some or all of these activities.
There is no cure, but treatments can help relieve symptoms. These may include breathing devices, physical therapy, and speech therapy.
ALS is rare, affecting between 1.5 and 3 in 100,000 people in North America and Europe.
Source: NHS and ALS Association
When Casey first participated in the BrainGate clinical trial, he had great difficulty understanding and needed others to help him interpret.
He had the BCI implanted in July 2023. Four microelectrode arrays were inserted into the left precentral gyrus – a brain region responsible for coordinating language – to record brain activity from 256 cortical electrodes.
“We can clearly see that they are trying to move their muscles and speak,” said Dr. Stavisky, assistant professor in the Department of Neurosurgery.
“We record the part of the brain that tries to send these commands to the muscles.
“And basically we listen in and translate those patterns of brain activity into a phoneme – like a syllable or a unit of speech – and then into the words they’re trying to say.”
Dr. Stavisky added, “It’s not that we read people’s minds, we perceive their inner thoughts or inner monologues or even the voice of their mind.”
“We can clearly see them trying to move their muscles and speak.”
“TRANSFORMATIVE TECHNOLOGY”
Previous BCI systems often had word errors that made it difficult for users to understand.
But within 30 minutes, using this latest technology, Casey achieved 99.6 percent word accuracy on a 50-word vocabulary.
“When we first tried the system, he cried with joy when the words he was trying to pronounce correctly appeared on the screen. We all did,” said Dr. Stavisky.
In the second session, the potential vocabulary increased to 125,000 words.
With only an additional 1.4 hours of training data, the BCI achieved a word accuracy of 90.2 percent, which later increased to 97.5 percent.
“Currently, we can correctly decipher what Casey is trying to say 97 percent of the time. That’s better than many commercially available smartphone applications that attempt to interpret a person’s voice (about 95 percent),” said Dr. Brandman.
“This technology is transformative because it gives hope to people who want to speak but can’t.
“I hope that technologies like this speech BCI will help future patients talk to their family and friends.”
The study reports 84 data collection sessions over 32 weeks.
In total, Casey used the voice BCI for over 248 hours in self-paced conversations to communicate in person and via video chat.
“It was immensely gratifying to see Casey regain his ability to communicate with his family and friends through this technology,” said the study’s lead author, Nicholas Card, a postdoctoral fellow in the Department of Neurosurgery at UC Davis.
8
8
How the technology works
Dr. David Brandman, a neurosurgeon at UC Davis Health, said, “Communication is so important to our daily lives.
“Something as simple as ‘I have an itch’ or ‘I’m thirsty’ or ‘I love you’ – imagine not being able to say those things to the people who mean the most in your life.
“A brain-computer interface is a device that records brain signals from paralyzed people and then translates those signals to enable people to communicate.
“This research is taking place as part of a clinical trial called BrainGate2.
“The idea is that people suffering from diseases such as spinal cord injuries, brain stem infarctions or ALS have electrodes surgically implanted into the motor cortex.
“They will then work with a scientific team to develop next-generation technologies to help paralyzed people.
“I implanted Casey Harrell, 45. He suffers from ALS and can hardly be understood when he tries to speak casually.
“By using a BCI, we can translate what he wants to say and make words appear on the screen.”
Neuroscientist Dr. Sergey Stavisky added: “Essentially, we are bypassing the injury.”
“We record the source, the part of the brain that is trying to send these commands to the muscles, and we translate these patterns of brain activity into phonemes — say, a syllable, the unit of language — and then into the words they’re trying to say.”
Dr Nick Card, postdoctoral fellow and lead author, said: “Not only can we accurately decode the words he reads from the screen, but we can also accurately predict what he wants to say in a free conversation.”
Dr. Stavisky said, “It’s not that we read people’s minds, we perceive their inner thoughts or inner monologues or even the voice of their mind.”
“We can clearly see them trying to move their muscles and speak.”
