Computer that can ‘read the mind’ helps man with motor neurone disease ‘speak’ again as family burst in tears

A MAN with a debilitating neurological disorder is able to 'speak' again thanks to a computer that translates brain activity into text that is read aloud.

Scientists have even been able to mimic Casey Harrell's voice using recordings from before he fell ill.

The dad, who has sensors implanted into his brain, said: "Not being able to communicate is so frustrating and demoralising.

"It is like you are trapped - often isolated and lonely.

"Something like this technology will help people back into life and society."

Heartwarming footage shows the 45-year-old with family as he tries the device for the first time.

"He started crying, I started crying, his family started crying," Dr Sergey Stavisky, a neuroscientist who oversaw the trial, said.

"Everyone had to stop to compose themselves. It was a really special moment."

Casey was diagnosed with amyotrophic lateral sclerosis (ALS) - a type of motor neurone disease (MND) - in 2020.

He started noticing symptoms after the birth of his daughter.

Initially, doctors suspected he had a muscle-twitching condition that would go away on its own.

Casey then tested positive for Lyme disease, which can mimic the early symptoms of ALS, before neurologists finally realised his limp and cramps were due to the muscle-wasting illness.

His condition continued to deteriorate, and he now uses a specialised chair.

Casey has lost the ability to stand and walk, and can no longer control the muscles used to speak.

But, scientists have developed a new brain-computer interface (BCI) that can interpret his brain signals when he tries to speak.

It then turns them into text that is ‘spoken’ aloud by the machine with 97 per cent accuracy.

It makes people cry who have not heard me in a while

Casey HarrellALS patient

Dr David Brandman, a neurosurgeon at UC Davis Health, said: "Our BCI technology helped a man with paralysis to communicate with friends, families and caregivers.

"Our paper demonstrates the most accurate speech neuroprosthesis (device) ever reported.";

The BCI even sounds just like Casey, as researchers used artificial intelligence (AI) to mimic his voice.

"By using video recordings that were collected prior to him having ALS, we were able to digitally reconstruct his voice," Dr Brandman said.

"So now, when he tries to speak, not only do words appear on his screen, but it sounds like him."

"It feels a lot like me," Casey, who is still able to work full time, said.

"It makes people cry who have not heard me in a while."

When Casey first enrolled in the BrainGate clinical trial, he was very hard to understand and needed others to help interpret for him.

In July 2023, he had the BCI implanted. Four microelectrode arrays were put into the left precentral gyrus - a brain region responsible for coordinating speech - to record brain activity from 256 cortical electrodes.

"We’re really detecting their attempt to move their muscles and talk," Dr Stavisky, an assistant professor in the Department of Neurological Surgery, said.

"We are recording from the part of the brain that’s trying to send these commands to the muscles.

"And we are basically listening into that, and we’re translating those patterns of brain activity into a phoneme — like a syllable or the unit of speech — and then the words they’re trying to say."

Dr Stavisky added: "It's not that we're reading people's minds, we're not detecting their inner thoughts, or inner monologue, or even their mind's voice.

"We're really detecting their attempt to move their muscles and to talk."

'TRANSFORMATIVE TECHNOLOGY'

Previous BCI systems had "frequent word errors", making it hard for users to be understood.

But within 30 minutes, Casey was using this latest technology to achieve 99.6 per cent word accuracy with a 50-word vocabulary.

"The first time we tried the system, he cried with joy as the words he was trying to say correctly appeared on-screen. We all did," Dr Stavisky said.

In the second session, the size of the potential vocabulary increased to 125,000 words.

With just an extra 1.4 hours of training data, the BCI achieved a 90.2 per cent word accuracy, which later rose to 97.5 per cent.

";At this point, we can decode what Casey is trying to say correctly about 97 per cent of the time, which is better than many commercially available smartphone applications that try to interpret a person’s voice [about 95 per cent]," Dr Brandman said.

"This technology is transformative because it provides hope for people who want to speak but can’t.

"I hope that technology like this speech BCI will help future patients speak with their family and friends."

The study reports on 84 data collection sessions over 32 weeks.

In total, Casey used the speech BCI in self-paced conversations for over 248 hours to communicate in person and over video chat.

"It has been immensely rewarding to see Casey regain his ability to speak with his family and friends through this technology," the study’s lead author, Nicholas Card, postdoctoral scholar in the UC Davis Department of Neurological Surgery, said.