Linking Perceived Surface Properties to Image Statistics

Michael J. Lander

Scientists have discovered a great deal about how the brain perceives objects. Less clear is how it identifies what these objects are made of based on their surface appearance. Understanding how humans make this determination would provide insights into brain function and could have applications in machine vision system design.

Edward H. Adelson and colleagues at MIT in Cambridge and at Nippon Telegraph and Telephone Corp. in Atsugi, Japan, have discovered a relationship that may help advance this understanding.

To begin the study, the researchers prepared textured stucco samples and coated them with paints of various shades and/or with clear acrylic media. With a Bitran 16-bit linear camera, the team photographed the objects and normalized the images’ mean luminance with computer programs to prepare them for analysis.

Immediately apparent to the investigators was that glossy samples seemed darker than matte ones with the same average luminance. In luminance histograms, these differences corresponded to a characteristic asymmetry that reflected the uneven arrangement of light and dark regions on the surface.

Dark and glossy stucco, for instance, displayed a positively skewed statistical distribution — one with an elongated right tail. Samples with a light and matte appearance had histograms with a relatively negative skew (see figure). Histograms with skewness values of zero display even tails on their right and left, and samples associated with them were expected to have intermediate characteristics.

To verify the correlation, the researchers showed human subjects images of stucco, black cotton fabric and crumpled white paper whose histograms they had computationally forced to have specific skewness values. Plots of the results showed that lightness ratings were negatively dependent and glossiness ratings positively dependent on skewness, reinforcing initial conclusions.

Skewness computation

Continuing their analysis of human perception, they proposed a neural processing chain that included skewness detectors, and they conducted a psychophysical experiment. In it, they asked subjects to focus on a point between negatively and positively skewed patterned images. When presented subsequently with a set of identical stucco images, subjects falsely identified them as having different lightness and glossiness, indicating their adaptation to pattern asymmetry.

Two pieces of textured stucco display different shades and glossiness, even though their mean luminance is equal. A comparison of luminance histograms shows that the distributions are noticeably skewed relative to one another. Courtesy of Edward H. Adelson, MIT.

“It seems plausible that the brain could compute something like skewness,” group member Lavanya Sharan said. She explained that the organ could do so using contrast polarity-sensitive cells that scientists have already identified. Currently, the researchers are expanding the study to include computer-rendered images and 3-D natural objects. These may enable them to more fully elucidate how humans recognize materials.

According to the team, an autonomous vehicle designed with similar recognition capabilities might be able to differentiate icy, wet and dry surfaces and to change its driving strategy accordingly. Robotic visual systems and surveillance cameras could collect more informative data. For these devices, computing skewness would prove much easier than doing inverse optics, which requires recovering the precise combination of surface structure, reflectance properties and surrounding illumination that gives rise to an image.

Furthermore, many surfaces possess asymmetry, making skewness assessments of potential use for surface property estimation in numerous environments.

Nature, May 10, 2007, pp. 206-209.