Xing-Fei He, Teledyne Dalsa Inc.
Multispectral imaging takes the detection capability of machine vision beyond just color inspection. With wafer-level coating of multispectral filters, advanced multispectral cameras now can be designed in a compact form factor. The result: many new opportunities in high-speed line-scan imaging.
In color imaging, objects that are indistinguishable in monochrome images are discriminated by color.1 Color itself is a useful characteristic, of course, but what if you could look beyond just color to see what is invisible to the naked eye – including security inks printed on currency and copper circuits buried within electronic circuit boards?
It’s already possible, and here’s how it works. In a silicon sensor, either CCD or complementary metal oxide semiconductor (CMOS) color filters generally are coated at the wafer level to select different wavelengths – i.e., red, green and blue (RGB). Optical images captured line by line at these wavelengths are then used to identify specific objects of interest. Spatial correction allows the camera to align all color channels at the same object point with subpixel accuracy.1 The advantage of line scanning over area scanning for color imaging is that it provides three native RGB colors without Bayer decoding, which gives the best color image quality.
For certain applications, color imaging is no longer enough. Applications that need new, more sophisticated multispectral imaging include currency inspection, electronics manufacturing and food sorting. These require specific wavelengths that are either outside the visible spectrum or in between the RGB color bands. These wavelengths produce unique optical characteristics for the object of interest. Multispectral imaging uses the same concept as color imaging but extends the wavelength from the conventional visible spectrum (RGB color bands) to, in most cases, NIR or ultraviolet regions. Recently developed thin-film interference technology enables advanced camera designs that provide spectrally independent outputs of RGB+NIR in a small form factor, opening up opportunities for machine vision applications.
Multispectral filter technologies
Different types of filter designs are available for multispectral cameras. Traditionally, a multispectral camera uses a prism-based beamsplitter to separate wavelengths. Images at different wavelengths are captured using multiple sensors, and the multispectral image is reconstructed later by combining these images. With a common optical axis, the prism-based camera captures simultaneously at the same location on the moving object. The drawbacks are increased camera costs and the need for more expensive optical lenses. Because a prism causes back focal shift and aberration, it needs a specially designed lens. The camera body is usually large enough to accommodate the prism and multiple sensors.
Dye and pigment are absorption-based filters coated on the silicon wafer. These are the most common materials used in color imagers today. Because most dye or pigment materials are transparent in the NIR, a dye- or pigment-based color camera generally requires an IR block filter to remove the IR contamination. Technical difficulties arise when the technology is extended to the NIR region because of the limited choices in organic materials. In certain camera designs, a global band-block filter separates the visible from the NIR. However, this requires digital subtraction of the NIR component from the RGB data and leads to inaccurate colors and higher noise.
Recently developed thin-film interference filter technology for the Piranha4 multispectral camera by Teledyne Dalsa relies on multilayer thin films to selectively pass light within a range of wavelengths while reflecting others. The technology overcomes the above limitations using organic materials. In the filter design, the bandpass wavelength window can be tailored to any specific spectrum required.
This multispectral camera is a quadlinear line-scan camera with 2K resolution and a 14.08 × 14.08-µm pixel size. Packaged in a small form factor of 62 × 62 × 48 mm, this camera outputs four channels of R, G, B and NIR and runs at a maximum line rate of 70 kHz with a Camera Link data interface.
Figure 1. Schematic of Teledyne Dalsa’s Piranha4 quadlinear multispectral CMOS sensor using thin-film interference filters. An IR filter blocks color channels, while a visible-block filter passes only NIR wavelengths.
Sensor design
Figure 1 shows a schematic of the sensor design using thin-film interference filters. One key feature of the technology is that the IR-block and visible-block filters are coated on a pixel level. For RGB channels, an IR-block filter is coated first and used to remove any IR components from the color image. For the NIR channel, a visible-block filter is implemented to allow only NIR signals to pass. This pixel-level thin-film architecture allows spectrally independent outputs from all RGB+NIR channels that capture accurate multispectral data without any digital manipulation.
The responsivity curves of Figure 2 show that each output of the four channels is independent. The RGB channels contain no NIR signals, and the NIR channel contains no visible light signals. The stop band in between 720 and 780 nm allows better isolation between the R and the NIR channels.
Figure 2. Responsivity curves of the Piranha4 quadlinear multispectral camera. With the wafer-level-coated IR-block and visible-block filters, the camera outputs show spectrally independent RGB+NIR data.
VIS+NIR
Historically, remote sensing has been the main market for multispectral imaging. The past few years, however, have seen increasing adoption of multispectral imaging for automatic optical inspection (AOI), including print inspection, electronics manufacturing, food and materials sorting, and medicine. Key requirements for industrial applications include high speed, low cost and a small footprint.
One emerging application is security printing for currency. ATM machines look for special security inks that cannot be viewed by the human eye, as an anticounterfeit measure. Figure 3 shows several images of a banknote captured by a multispectral camera. The images include (a) color, (b) monochrome and (c) NIR. The euro banknote uses specific security inks that are visible only under lighting around 850 nm in the NIR region. At a commercial printing company, the quality of security inks is 100 percent inspected during the printing process. Previously, system designers needed a single color camera to inspect the banknote’s color and a separate NIR camera to inspect the security inks, complicating the designs and adding costs.
Figure 3. A euro banknote image captured by the Piranha4 multispectral camera. Three different outputs: (a) color, (b) monochrome and (c) NIR. Light sources were white LEDs for color and monochrome and 850-nm LEDs for NIR. The image of architectural windows in the banknote was half-printed with IR security inks. The security inks are invisible to the naked eye, as shown in the color and monochrome images. With the NIR light source at 850 nm, the NIR channel unveils these invisible features.
With a multispectral camera, system designers can use a single camera with the combination of white and NIR LED light sources. In this case, the multispectral camera dramatically simplifies system designs and reduces costs.
Figure 4 shows a high-speed vision system developed by CBPM-Great Wall Financial Equipment Holding Co. Ltd. of Beijing. The system uses a multispectral RGB+NIR camera to inspect both color and NIR images printed with security inks on the banknotes.
Light sources
Lighting plays an important role in multispectral imaging. Today, LEDs are the most common, most efficient light sources. For visible light, white LEDs are used primarily for monochrome and color imaging. Because light from a white LED contains very little IR component, a NIR LED must be added for RGB+NIR imaging. The choice of wavelength is also important to reveal specific features of interest. A NIR light with wavelengths of 780 to 1050 nm can be used, depending on application requirements.
Figure 4. This banknote inspection system uses Teledyne Dalsa’s Piranha4 multispectral camera to capture accurate RGB+NIR images at high speed, enabling inspection of both color and NIR images within one camera. Photo courtesy of CBPM-Great Wall Financial Equipment Holding Co. Ltd.
In certain applications, UV excitation is used to inspect fluorescent inks, another type of security ink. The Piranha4’s filter is designed to carefully cut off UV light below 380 nm, so the excitation cannot leak into the visible images.
Imaging from surface to depth
Although humans can see only what is visible on the surface, imaging technology using different wavelengths can “drill down” and see what is hidden at various depths. Imaging technology detects optical reflection from a surface, and the optical properties can change significantly with the wavelength of the light source. Light penetration depth of many materials depends on the wavelength, so a selection of wavelengths can capture images of features at different depths.
Electronic circuit manufacturing has started to adopt multispectral technology for imaging at different depths. In a finished printed circuit board (PCB), colored paints are often used as a top coating. All of the copper wires and paints need to be inspected during the manufacturing processes. Today, color imaging is a main tool for final inspection of PCBs.
Figure 5. Images of a printed circuit board taken using light sources of various wavelengths: (a) UV at 400 nm, (b) visible using a white LED and (c) NIR at 850 nm. UV light reveals only features on the surface, but not deep inside the board. Copper wires underneath the paint clearly appear in the NIR image, where light with longer wavelengths penetrates more deeply.
Figure 5 shows three images of a PCB captured using UV, visible and NIR light sources. The soldering pads, finished paints and wires below are all objects of interest for the AOI. In Figure 5a, the shorter-wavelength UV light provides little penetration, revealing only the soldering pads and surface morphology. It doesn’t show any copper wires. In Figure 5b, visible light captures the surface and, to some extent, the wires below the paint. But in Figure 5c, the longer-wavelength NIR light offers greater penetration depth, clearly revealing the copper wires covered by the paint. Also, copper happens to have higher reflectivity in NIR regions, so these wavelengths obtain better image contrast. Clearly, the images in Figure 5 indicate that by choosing the right wavelength, you can selectively image the surface or the wires underneath the paints, as well as other features buried deeper in the board. v
In this case, a multispectral camera that provides UV+VIS+NIR or UV+RGB+NIR output channels would be ideal for the AOI. With this combination, a single camera could inspect defects at different depths within the PCB. For example, one channel could inspect surface defects, and another could detect buried copper wires. The camera can be realized using a multiline sensor architecture. However, finding an optical lens that covers the wide spectrum from UV to NIR could be a challenge.
Meet the author
Dr. Xing-Fei He is senior product manager at Teledyne Dalsa Inc. in Waterloo, Ontario, Canada; email: [email protected].
Reference
1. X.F. He (2013). Trilinear cameras offer high-speed color imaging solutions. Photonics Spectra, Vol. 47, Issue 5, pp. 34-38.