Polarization Cameras Get Ready for Prime Time

HANK HOGAN, CONTRIBUTING EDITOR

Polarization can remove glare in vision systems and improve imaging through transparent materials such as glass. This capability is beneficial for surface inspection of film thickness or tilt angle. It is also critical for spotting contaminants in products, when contaminants closely resemble the contents of plastic bins or impurities in pharmaceuticals.

Inspecting for impurities can often be accomplished by distinguishing color differences, according to Kevin Harding, president of the consulting firm Optical Metrology Solutions. But when this doesn’t work, polarization can be effective, he said.

Standard color imaging reveals a vehicle obstructed by foliage. Courtesy of Teledyne FLIR.

The arrival of new cameras based on recently available polarization sensor technology from Sony could make implementing such glare-reducing inspection techniques easier and less costly. These innovations could help move polarization sensing machine vision from a niche market into the mainstream, though education and training in the capabilities of the technology will play a role, said Harding.

Extracting polarization states

Polarizing cameras extract three to four polarization states of incoming light, and many can do so on a pixel-by-pixel basis. The polarization could be, for example, along a 0°, 45°, 90°, and 135° direction.

“In doing that, you actually have a lot more information than you had before,” Harding said.

Using three- or four-state polarization data, it is possible to calculate a number of parameters that relate to the surface material of an object. It then becomes easier to spot surface differences.

For example, the polarization of the light reflected off a person differs from that coming off of a tree or car. This type of information from reflected light is valuable in surveillance when trying to distinguish between objects that are at the limit of resolution and, therefore, only a few pixels in size. A similar approach could be used when performing an inspection in a factory to differentiate between objects that otherwise appear similar due to resolution constraints.

For transparent objects, polarization can reveal internal characteristics. Stress differences in plastic can sometimes change the polarization state of light passing through one region of a part compared to another region. These differences provide information about the processing that produced the part.

The number of applications that actively use polarization data is still quite limited. One reason for this is a lack of training and education, and another is the lack of easy-to-use polarization sensing technology.

Traditionally, system builders used fixed filers or a filter wheel to achieve polarization sensing. The former provides polarization in only one direction, that of the fixed filter. This approach is relatively simple, but it is only helpful for a restricted number of cases. In a filter wheel, in contrast, imaging occurs successively, and the filter is moved after each polarization state capture. The filter wheel method is more general purpose but also more expensive to implement and slower than the fixed-angle filter technique.

Sony sensor advancements

The advent of new sensor technology offers a third avenue to polarization information. Sony debuted a polarization sensor in 2018 that included 5-MP versions in color and monochrome. The products were based on the company’s line of CMOS visible spectrum sensors.

On the polarization sensing chip, the pixels are arranged in 2 × 2 blocks that are repeated to make up the entire sensor. Within each block, polarization filters arranged at four different angles sit atop the pixels, ensuring that each pixel preferentially picks up light polarized along its respective direction. In effect, the chip contains four fixed, pixelated polarization sensors.

Thus, no external filter is needed, and a filter wheel is also unnecessary. Camera makers have since released products based on the sensors. Examples include SVS-Vistek and Teledyne FLIR.

Pixelated polarization sensing can enhance differences and improve contrast (the same scene as depicted in color on previous page). Courtesy of Teledyne FLIR.

One key benefit to using a polarized sensor is that it reduces reflection.

“This can be very useful whenever [you’re] trying to image through a surface that may cause reflections,” said William Gallego, Teledyne FLIR product marketing manager. “For example, if a camera is trying to review the contents found in a clear plastic bag, or a traffic camera [is] attempting to take photos of license plates or [it’s] imaging through a car windshield, such as for HOV [high-occupancy vehicle] or seat belt infractions,” he said.

Due to reflections, standard RGB or monochrome visible-spectrum machine vision cameras may capture absolutely no details in some areas of the image. A polarized sensor camera, however, is able to reduce glare to capture otherwise invisible features.

In practice, this process begins when the camera captures images and sends the data to a PC. There, software receives the data and sends it through a polarization algorithm to translate the raw input into a new format, such as a glare-reduced version.

The algorithm can also extract the Stokes parameters — four values that describe the polarization state of light. These parameters can be correlated to surface roughness and composition.

Using polarization data, cameras reject glare and improve imaging. Original (left) and glare-reduced image (right). Courtesy of Optical Metrology Solutions.

A third possibility is to turn each captured image into four images, one for each of the polarization filters. Gallego said an algorithm could interpret the data in other ways as well. This image-conversion process takes computing power and time, due to the added steps. The process is similar to that of converting a raw image to JPEG format, he said.

The extra information gained by capturing light’s polarization state comes with downsides. Since the sensor arranges pixels in 2 × 2 blocks, resolution may effectively only be a quarter of that for the full chip because only 25% of the pixels capture the same polarization state and therefore “see” the same scene.

Drop in light intensity

Some light loss is a consequence of sending incoming photons through a polarization filter. The filters have a 4:1 extinction ratio, so some light gets through even if the polarization state of the light does not align with the polarization angle of the filter. But some incoming light will be blocked by the filter. Sensor quantum efficiency measurements performed by Teledyne FLIR, according to the EMVA 1288 specification, indicate that there may be a two- to three-fold drop in light intensity for polarization sensors when compared to sensors without the capability.

The current sensor, which is from Sony, is more costly than chips with an equal number of pixels that do not sense polarization. Gallego said Teledyne FLIR is investigating less expensive alternatives.

The company is also looking into deploying a higher-resolution version of the sensor in a new camera. While the current cameras use 5-MP sensors, the new one may use a 12-MP sensor.

Changes in polarization indicate areas of differing stress in plastic. Courtesy of Optical Metrology Solutions.

Today’s polarization sensing cameras can provide significant benefit in some applications, such as performing inspections through the windshield of a car to see whether seat belts are being worn by a driver or passengers. The cameras must be able see through windshields regardless of the level of glare, which varies by time of day and season.

For transparent objects, polarization can reveal internal characteristics. Stress differences in plastic can sometimes change the polarization state of light passing through one region of a part compared to another region. These differences provide information about the processing that produced the part.

While the sensor is critical to the operation of these cameras, it is not enough by itself.

“Beyond the sensor itself, it was important to add in the necessary functions to interpret the polarization data,” Gallego said. “Without the polarization algorithms we incorporated into our software, other third-party software would be unable to convert the raw polarized image into useful data.”

Finally, discussing the possible future for polarization sensing cameras, Harding recalled the past. Thirty years ago, he and other machine vision experts were happy with a 512K-pixel image even if it was monochrome and the equipment was expensive, he said. Today, 10-MP full-color cameras cost a few hundred dollars, and machine vision applications have benefited from the 20-fold increase in resolution and a substantial decrease in price. Polarization sensing technology may follow a similar trajectory.

Imaging through glass in variable natural lighting is difficult due to reflections and glare (top), but imaging is easier when cameras use polarization data to remove reflections and glare (bottom). Courtesy of Teledyne FLIR.

Harding cautioned, however, that knowing how to use a tool is an important first step in getting it into vision applications. Thus, training and education about how to use a polarization camera may be as necessary as the hardware and software.

Given the right mix of training and technology, the new point of view allowed by these machine vision cameras could bring substantial benefits. Focusing on polarization sensing, Harding said, “There certainly are a lot of possibilities where it would be very helpful.”