Compact single-shot metalens depth sensor inspired by eyes of jumping spiders
Qi Guo* Zhujun Shi* Yao-Wei Huang Emma Alexander Cheng-Wei Qiu Federico Capasso Todd Zickler*Equal contributions
Abstract: Jumping spiders (Salticidae) rely on accurate depth perception for predation and navigation. They accomplish depth perception, despite their tiny brains, by using specialized optics. Each principal eye includes a multi-tiered retina that simultaneously receives multiple images with different amounts of defocus, and from these images, distance is decoded with relatively little computation. We introduce a compact depth sensor that is inspired by the jumping spider. It combines novel metalens optics, which modifies the phase of incident light at a subwavelength scale, with efficient computations to measure depth from image defocus. Instead of using a multi-tiered retina to transduce multiple simultaneous images, the sensor uses a metalens to split the light that passes through an aperture, and concurrently form two differently-defocused images at distinct regions of a single planar photosensor. We demonstrate a system that deploys a 3mm-diameter metalens to measure depth over a 10cm distance range, using fewer than 700 floating point operations per output pixel. Compared with previous passive depth sensors, our metalens depth sensor is compact, single-shot and requires a small amount of computation. This integration of nano-photonics and efficient computation brings artificial depth sensing closer to being feasible on millimeter-scale, micro-Watts platforms such as micro-robots and micro-sensor networks.
Significance statment: Nature provides diverse solutions to passive visual depth sensing. Evolution has produced vision systems that are highly specialized and efficient, delivering depth perception capabilities that often surpass those of existing artificial depth sensors. Here, we learn from the eyes of jumping spiders and demonstrate a metalens depth sensor that shares the compactness and high computational efficiency of its biological counterpart. Our device combines multifunctional metalenses, ultrathin nanophotonic components that control light at a sub-wavelength scale, and efficient computations to measure depth from image defocus. Compared with previous passive artificial depth sensors, our bioinspired design is lightweight, single-shot and requires a small amount of computation. The integration of nano-photonics and efficient computation establishes a new paradigm for design in computational sensing.
Publication
Qi Guo, Zhujun Shi, Yao-Wei Huang, Emma Alexander, Cheng-Wei Qiu, Federico Capasso, Todd Zickler. "Compact single-shot metalens depth sensors inspired by eyes of jumping spiders." Proceedings of the National Academy of Sciences (PNAS), 2019.
[pdf] [supplement] [demo video] [DOI]
Media coverage
4 Awesome Discoveries You Probably Didn't Hear About - Episode 37. National Science Foundation.
Inspired by jumping spiders. Harvard Gazette.
Compact depth sensor inspired by spiders. Harvard SEAS.
Spider eyes inspire a new kind of depth-sensing camera. TechCrunch.
An efficient new depth sensor was inspired by spider eyes. Engadget.
Jumping Spiders Inspire Incredible Compact Depth Sensor. SciTechDaily.