This 4D Camera Will Help Robots Better Navigate Their World

Your eyes are great. They can look around, adjust to the light in the room, and focus on the things you find important. Robots, on the other hand, use cameras to see. Cameras can do a lot of the same things our eyes can, but robots aren't like us — their processors let them take in more information in less time. That's why engineers have created a camera that lets robots generate information-rich images in four dimensions.

Assistant Professor Gordon Wetzstein, left, and postdoctoral research fellow Donald Dansereau with a prototype of the monocentric camera that captured the first single-lens panoramic light fields.

Take It All In

"We're great at making cameras for humans, but do robots need to see the way humans do? Probably not," says Stanford engineer Donald Dansereau. To create a robot-centered camera, Dansereau, Gordon Wetzstein, and a team from the University of California, San Diego dusted off 20-year-old technology known as light-field photography. This type of photography captures images a lot like a standard camera, but also collects information about the direction and distance of the light rays in the scene. That way, you can refocus an image after it's taken, meaning that an autonomous vehicle could look past drops of rain on a stormy drive and a drone could focus on individual obstacles as it makes its way through a cramped store aisle. Right now, robots have to move around to get enough information about a scene, but light-field technology lets them take it all in with just a glance.

The camera's field of view is impressive, too. At 140 degrees, it encompasses nearly a third of the space around a camera. That presented a challenge to the researchers, however, since that requires a spherical lens but robots have flat sensors. The two university teams combined their expertise to create a digital solution that actually resulted in better images overall.

Off-The-Shelf, But Better

If the technology sounds familiar, that's because there have been consumer-level light-field cameras on the market since 2011, starting with the first Lytro camera. "Many research groups are looking at what we can do with light fields but no one has great cameras. We have off-the-shelf cameras that are designed for consumer photography," said Dansereau. "This is the first example I know of a light field camera built specifically for robotics and augmented reality. I'm stoked to put it into people's hands and to see what they can do with it."

The camera is still a proof of concept, and it's not yet small enough to use with a robot. But the researchers plan to make a more compact version next, which could be small and light enough to use in robots and even wearables, like virtual- and augmented-reality headsets.

How Does a Light Field Camera Work?

Written by Ashley Hamer August 10, 2017

Curiosity uses cookies to improve site performance, for analytics and for advertising. By continuing to use our site, you accept our use of cookies, our Privacy Policy and Terms of Use.