Breakthrough allows brain signals to be turned into speech

Author: Mark Gould
Breakthrough allows brain signals to be turned into speech

Scientists have developed a brain implant that can read people's minds and turn their thoughts to speech. They say the implant could help people who have lost their ability to speak due to Parkinson's, stroke, brain injuries or throat cancer. There is also the more distant possibility of helping people who have never spoken, including some children with cerebral palsy, to learn to speak with such a device, say researchers.

The team at the University of California, San Francisco writing in the journal Nature* implanted an electrode into the brain to pick up the electrical signals that manoeuvre the lips, tongue, voice box and jaw. Then computing is used to simulate how the movements in the mouth and throat would form different sounds. This results in synthesised speech coming out of a "virtual vocal tract".

Focusing on the shape of the mouth and the sounds it would produce have allowed the scientists to achieve a world first. The system is better with prolonged sounds like the "sh" in ship than with abrupt sounds such as the "buh" sound in "books".

In experiments with five people, who read hundreds of sentences, listeners were able to discern what was being spoken up to 70% of the time when they were given a list of words to choose from.

The team say the procedure could work for patients with many conditions but the technology relies on the parts of the brain which control the lips, tongue, voice box and jaw working correctly. So, patients with some types of stroke would not be able to benefit.

Professor Edward Chang, one of the researchers, said: "For the first time, this study demonstrates that we can generate entire spoken sentences based on an individual's brain activity.

"This is an exhilarating proof of principle that, with technology that is already within reach, we should be able to build a device that is clinically viable in patients with speech loss."

The participants in the study were told not to make any specific mouth movements. Professor Chang said they were just asked to do the very simple thing of reading some sentences.

"So it's a very natural act that the brain translates into movements itself."

Professor Chang said: "We and others actually have tried to look at whether it's actually possible to decode just thoughts alone.

"And it turns out, it's a very, very difficult and challenging problem.

"That's only one reason why, of many, we really focus on what people are actually trying to say."

However, some scientists have argued there is an ethical debate to be had about brain-machine interface technologies that read the mind.

*Guglielmi G. Brain signals translated into speech using artificial intelligence. Nature, 24 April 2019. DOI: 10.1038/d41586-019-01328-x