Science & Technology

Scientists Taught an Algorithm to Translate Thoughts Into Speech

In "What Women Want," protagonist Nick Marshall gets electrocuted in a hairdryer-in-the-bathtub mishap. Suddenly, he can hear women's thoughts as if they were saying them aloud. It's not exactly a classic, but this film from 2000 has staying power — not just because it was recently remade, but because it's getting closer to reality. Recently, for the first time ever, neuroengineers at Columbia University made human thought into intelligible, audible speech.

Related Video: Researchers Demonstrate 'Mind-Reading' Brain-Decoding Tech

Siri the Mind Reader?

The thought-to-speech breakthrough, detailed in a recent paper from researchers at Columbia University and Hofstra/Northwell School of Medicine, relies on the same technology that powers our robot butlers Siri and Alexa: the vocoder. A vocoder is just a computer algorithm that can synthesize human speech, and in this case, the research team taught such an algorithm to recognize and translate brainwave patterns in a person's auditory cortex into speech. The auditory cortex is the area of the brain that's activated when we speak and listen, and — importantly! — when we imagine speaking and listening. In other words, when we think.

Well, actually — pause for a second. The definition of thought is an open philosophical question. It could be considered everything a conscious brain does. In the context of this study, though, it's "inner speech," or language we've actively called to mind. Conscious brains do many other things, too: They can also, for instance, picture a sunflower, remember a smell, or perform mental math. But words constitute a substantial chunk of our waking thoughts — at least 20 percent of them, by psychologist Charles Fernyhough's estimate — and verbal thoughts were the focus here.

To teach the vocoder to translate verbal thoughts, researchers relied on brain scans conducted during routine brain surgeries on epileptic patients. They tracked these patients' auditory cortex activity as they listened to sentences read aloud by a variety of people. Over time, the scans taught the vocoder to match neural activity to simple speech. So far, it can "read" the numbers 0 through 9 from brain activity alone; next, researchers plan to experiment with other words and sentences.

That's only part of the breakthrough here, though. It's also groundbreaking that the vocoder's robo-voice can translate brainwaves reliably and intelligibly. "We found that people could understand and repeat the sounds [of the vocoder] about 75 percent of the time, which is well above and beyond any previous attempts [at translating thought]," said one of the study's co-authors.

A Voice for the Voiceless

Sure, it's not as intelligible as a human voice yet, but this new technology still has huge implications — and not just for watchdogs fact-checking "What Women Want." This technology could one day offer a new way to connect with the world for anyone who can't speak, whether it's due to a stroke, amyotrophic lateral sclerosis (ALS), or some other illness or injury.

The research remains in the early stages, but researchers hope that one day, this vocoder technology can be built into an implant — something akin to the chest implant some epilepsy patients wear to reduce the frequency and severity of their seizures. Just like a voice, this implant would allow patients to communicate just by thinking. No pressing of buttons, no writing of notes, not even a need for an eye-tracking interface like Stephen Hawking used.

What about privacy? A device that broadcast your every thought would streamline communication while creating a million other embarrassing problems. Thankfully, the implant researchers envision would only translate thoughts on command. Still, as convenient and groundbreaking as the device would be, it does open up the possibility of hacking into another human's thoughts — which sounds at the same time alarming, epic, and, if Twitter is any indication of what thought looks like, pretty dull.

Get stories like this one in your inbox or your headphones: Sign up for our daily email and subscribe to the Curiosity Daily podcast.

Want to sharpen your mind? Check out "Smarter: The New Science of Building Brain Power" by Dan Hurley. We handpick reading recommendations we think you may like. If you choose to make a purchase, Curiosity will get a share of the sale.

Written by Mae Rice February 13, 2019

Curiosity uses cookies to improve site performance, for analytics and for advertising. By continuing to use our site, you accept our use of cookies, our Privacy Policy and Terms of Use.