Abstract: We develop a framework for extracting a concise representation of the shape information available from diffuse shading in a small image patch. This produces a new mid-level scene descriptor, comprised of explicit local shape distributions that are inferred separately at every image patch across multiple scales. Our framework is based on a quadratic representation of local shape that, in the absence of noise, has guarantees on recovering accurate local shape and lighting. And when noise is present, the inferred local shape distributions provide rich shape information without over-committing to any particular image explanation. As an example of their utility, we show that these local shape distributions enable efficient and robust reconstruction of scene lighting and object-scale shape, by naturally encoding the fact that some smooth diffuse regions are more informative than others.Publication
Ying Xiong, Ayan Chakrabarti, Ronen Basri, Steven J. Gortler, David W. Jacobs, and Todd Zickler, "From Shading to Local Shape", arXiv:1310.2916 [cs.CV], 2013.Results
Please click here for detailed results with interactive visualizations.Code
Please click here for our reference implementation and documentation on how to use it.