American scientists have created a technology that aims to translate thoughts into intelligible speech. The device reads the signals of brain activity in the moment when one wants to say anything. To develop the device that connects the possibilities of speech synthesis and artificial intelligence, is an important step in improving interfaces “brain — computer”. In the future, this invention will be able to use people who have lost the ability to speak as a result of injury or disease.
Scientists from Columbia University (USA) have created a technology that can translate human thoughts into intelligible speech. For this, the researchers turned to the vocoder — a computer algorithm that can synthesize speech after “training” after listening to recordings of people talking.
“It’s the same technology they use Amazon’s Echo and Apple’s Siri to verbally answer our questions,” explained study author Nimes, Masharani.
First, the experts needed to teach the vocoder correctly interpret the brain activity of a person. For this, the algorithm had to engage in work with very accurate data.
“We asked patients with epilepsy who are undergoing surgical treatment on the brain, listen to suggestions, which is pronounced by different people. At the same time, we analyzed patterns in the brain activity of patients. The obtained neural models and studied the vocoder,” he said, Masharani.
The researchers then asked the same patients to listen to the recording, which sounded of a sequence of digits from 0 to 9. At the same time, the scientists recorded brain signals, and then passed it through a vocoder. Neural networks — a kind of artificial intelligence that mimic the structure of neurons in the human brain, has analyzed the sounds that the vocoder was issued in response to these signals. As a result, you could hear a robotic voice, repeating a sequence of numbers. To check how clear it turned out the speech of the robot, the engineers asked volunteers to listen to her.
“We found that people can recognize the words about 75% of the cases, significantly superior to any previous attempts. Delicate vocoder and powerful neural network generated sounds that are listened to patients with surprising accuracy,” he said, Masharani.
In the near future, the researchers plan to teach artificial intelligence to pronounce more complex words and even whole sentences. After that, they intend to incorporate the invention in the implant, which will translate human thoughts into speech.
“In that case, if the owner of the implant will think: “I need a glass of water” — our system considers the brain signals and turns them into speech. It would give anyone who has lost the ability to speak due to injury or illness, a new chance to communicate with the outside world”, — concluded Masharani.
Read more •••