Science & Technology

This New Algorithm Can Read Your Brainwaves to See What You're Seeing

There's an episode of "Black Mirror" where an insurance agent investigating a hit-and-run interviews witnesses with a a device that reads their minds and replays their memories on a computer monitor. Now, new developments have us hoping the real thing works out better than the TV version. Researchers have created an algorithm that can reconstruct what you're seeing by reading your brainwaves.

Dan Nemrodov (left) and Professor Adrian Nestor (left) have developed a technique that can harness brain waves gathered by data to show how our brains perceive images of faces.

Look Into My Thoughts

This isn't even the first time scientists have read people's thoughts to see what they were seeing: in 2016, researchers used fMRI brain scans to do just that. The important difference in this breakthrough is that while fMRI requires volunteers to get inside a large machine and lie very still for minutes on end while it scans their brain for blood flow, this new breakthrough just requires you to wear a special EEG cap. EEG stands for electroencephalogram, and it uses electrodes to analyze your brainwaves. (The new research is published in the journal eNeuro.)

EEG has other benefits, too. "fMRI captures activity at the time scale of seconds, but EEG captures activity at the millisecond scale," said Adrian Nestor, assistant professor at University of Toronto Scarborough whose lab developed the new technology. "So we can see with very fine detail how the percept of a face develops in our brain using EEG."

Words Can't Describe

For the study, each volunteer put on an EEG cap and looked at images of various faces. The EEG equipment then recorded their brain activity, which machine-learning algorithms analyzed to actually reconstruct the image from the perceptions in the volunteer's mind. Here's what the algorithms created:

Examples of images extracted from EEG-based face-space constructs for (A) neutral and (B) happy faces.

It's not exactly high-definition footage, but it's an amazing start. The team's next step is to try and reconstruct images not just from what someone's eyes are currently seeing, but from a memory formed in their mind. That could be huge. "It could provide a means of communication for people who are unable to verbally communicate," said Nestor. "Not only could it produce a neural-based reconstruction of what a person is perceiving, but also of what they remember and imagine, of what they want to express."

And, of course, there's the law-enforcement angle. Instead of relying on sketch artists and police lineups, a real-life version of a Recaller could tap into a witness's memory and reconstruct what they saw. Forget security-camera footage — cops just need your thoughts.

How Harnessing Brain Waves Can Reconstruct Mental Images

Written by Ashley Hamer April 2, 2018