New Imaging Technique Reduces Time to Failure in Auto Paint, Cosmetics

JEREMY LERNER, LIGHTFORM INC.

Whether it’s metallic automotive paints, cosmetics or even coatings used in electronic displays, there’s more than meets the eye when it comes to coatings and complex composites. That’s because these materials are composed of diverse components including dyes, nanoparticles, macro-particles and pigments. Problems occur when any or all of the component materials physically or chemically interact in unexpected ways. This can be because of errors in process, mixing, deposition, variations in temperature and changes in pH or ion concentration.

A commonly used method for characterizing surfaces is multispectral imaging (MSI). The challenge: MSI typically does not deliver analytical spectroscopic data. It uses relatively few, often noncontiguous, wavelength data points.

Where greater sensitivity and specificity is required, a growing number of industries are turning to hyperspectral imaging microscopy (HSIM). HSIM provides a true analytical spectroscopic analysis that can identify abnormal changes in spectra within the field of view (FOV) and could predict possible manufacturing or reduced “time to failure” problems. The major benefit of HSIM is its ability to provide a detailed analysis of surfaces at a submicron level. This is often where failures in processes and materials start and easy visualization is not possible. The mixing of paints and composites in the automotive and cosmetic industries are good examples of this.

Given that most, if not all, materials involved are known, it is possible to record their spectral characteristics to create a reference spectral library (RSL). This library can then be used to compare and contrast spectra presented by objects in the FOV. If components cluster, bind or in any way interact either physically or chemically, their spectral characteristics will often change and mark a starting point for a future failure. Hyperspectral imaging considers each component in the FOV that presents a spectrum to be a spectral object, which can be compared to spectra in the RSL.

Wavelength dispersive HSIM provides the means to determine ratios of components and quantify their deviation from the standard. Understanding the interactions that may occur at a micron or submicron scale greatly increases the chances of identifying the root cause of a problem and can suggest a solution. All-natural products and those extracted from natural products used in production suffer, for example, from the variabilities of nature. This can affect adhesion, physical and chemical properties, and the absorption of dyes. Problem areas are likely to be heterogeneously distributed across a surface and may not be obvious by visual inspection.

The increased use of nanoparticles presents its own challenges. By definition, nanoparticles are smaller than the wavelength of light, so it is not possible to see or resolve them. However, using HSIM in dark-field spectroscopic scatter interactions between nanoparticles can be readily detected in aggregation, clustering and variations in size and shape.

The practical implementation of noncontact, spectral mapping (also referred to as spectral imaging and hyperspectral imaging), MSI and spectral topographical mapping began in the early 1970s with NASA and the U.S. Geological Survey. Imaging spectroscopy systems were interfaced with a telescope mounted on an aircraft or satellite for remote sensing. Concurrently, the chemistry community developed imaging Raman and infrared instruments for spatially resolved distribution of molecular species and crystal characteristics. The solution to characterizing heterogeneous solids at micron to submicron spatial resolution was found by repurposing remote spectral imaging from a telescope to any research microscope. This approach is now being used in industry, agriculture and medicine for quality control and research.

Multispectral vs. hyperspectral imaging

MSI characterizes a fixed FOV in a noncontiguous series of wavelength bandpass filters commonly produced with a liquid crystal tunable filter, or an acousto-optic tunable filter coupled to a CCD camera. Each bandpass filter, typically delivering from 10 to 20 nm per bandpass, contributes one wavelength data point.

An FOV that could be characterized by 50 wavelength data point spectra would be acquired 50 times sequentially, and a 50 data point spectrum would accumulate in each camera pixel. Effectively, a “photograph” would be taken through each bandpass filter to generate a series of wavelength slices through the FOV. Data sets accumulate to form a data cube where the height and width of the FOV is in X and Y, and wavelength in Z. In order to generate a complete spectral characterization, a scan must be fully completed and the FOV rigidly fixed to preserve spatial/wavelength registration.

How does HSIM work?

HSIM uses a prism (or diffraction grating) imaging spectrophotometer coupled with a CCD camera to work from 365 to 920 nm simultaneously, with a bandpass varying from 1 to 5 nm. An unlimited FOV is characterized sequentially by many hundreds of contiguous wavelength spectra. HSIM also provides analytical and spectroscopic functions that include absorption, percent reflection, fluorescence, luminescence and emission. Sample spectra are distributed along rows of pixels in X, and spatial information along columns of pixels in the Y direction; there is no Z.

HSIM works by mounting a sample on a computer-controlled microscope translation stage. A microscope objective then projects a magnified image of the FOV onto the entrance slit of the spectrophotometer. The microscope translation stage may need to be very large and in some cases incorporated into software — such as LightForm’s PARISS (Prism and Reflector Imaging Spectroscopy System) — that can even control a robotic sample handling system. Customization of the majority of HSIM systems is not necessary apart from sample handling (Table 1).

Sample spectra are correlated with those in the RSL. Individual spectra in this library are uniquely pseudo-colored with colors that may not necessarily be present in the original sample.

The software enables a scan of an unlimited FOV that does not require constructing a data cube. During sample translation, a sequential series of independent spectral characterizations are acquired across physical slices of the FOV, and a field scan can be terminated at any time without losing spectral integrity or registration. A single spectral snapshot across a target area of the FOV can be taken in milliseconds, capturing objects in motion with analytical accuracy. However, this mode is a challenge or impossible with data cube methodology.

Following a field scan, a grayscale spectral image can be generated by summing intensity along rows of pixels. Sample spectra that correlate with RSL spectra that meet or exceed a minimum correlation coefficient are then “painted” onto the grayscale image with pixel-perfect registration. Spectra that fail to correlate remain in gray.

The flow of this process involves the described system, which defines the boundaries and characteristics of spectral objects close to the component level (Figure 1). Specifically, the system can identify early signs of defects; define areas of interest; count spectral objects and quantify their statistical and spatial distribution; confirm the makeup and spatial distribution of color centers in paints and powders; characterize the emission characteristics and distribution of electronic emitters such as light emitting arrays and the photo-active surface films used in solar panels; and determine the size, shape and distribution of nanoparticles. Once a field scan has been acquired, software joins individual spectral slices sequentially to create the hyperspectral image and provide near real-time output.

Figure 1. This example of a hyperspectral microscope assembly includes a reflection/transmission microscope; a PARISS prism spectrophotometer, which operates from 365 to 920 nm simultaneously; and an observed image camera. Demonstrated above the assembly is a metallic automobile paint sample at 20× magnification (a). A gray-scale hyperspectral image (b). In a reference spectral library, all spectra are individually pseudo-color-coded by the user (c). Hyperspectral image in pseudo-colors of an area 0.25 × 0.25 mm in the field of view; it has been sampled 34,669 times (d). A spectral histogram of 34,669 spectral data points, each characterizing an area of 1.25 × 1.25 µm (e). Demonstrations of CIE (International Commission on Illumination) color space in an acquisition (f). Courtesy of LightForm Inc.

Spectral objects consisting of dyes and solid pigments in a heterogeneous metallic paint or cosmetic sample can be characterized in percent reflection (Figures 2, 3). Spectral objects can be delineated, counted and presented in CIE (International Commission on Illumination) color space.

Spectrometer and microscope parameters that impact performance

Constraints of a wavelength dispersive spectrometer can be seen in the impact of the width of the entrance slit of the imaging spectrometer on spatial and spectral resolution; this cannot be over-emphasized. The microscope objective projects a magnified image of an area of the FOV onto the entrance slit, overfilling it. The section of the image that passes through the slit to the spectrophotometer is an image of a physical slice of the FOV.

Figure 2. This graph indicates the percent-reflection plots of two RSL spectra and an unknown spectrum observed, as shown in the hyperspectral image in gray. Courtesy of LightForm Inc.

Spectral objects in the slice are arrayed along the height of the slit. As the sample is translated, all spectral objects in the entire FOV will pass through the slit sequentially, with the spectra acquired simultaneously. The entrance slit is imaged onto the pixels of the CCD camera at each wavelength; three pixels should fill the width of the slit’s image.

Efficiency, spectral resolution, bandpass

To maximize light throughput and signal-to-noise (S/N) ratio with a hyperspectral microscope, the described system features a prism with a flat 90 percent transmission efficiency from 365 to 920 nm. Software such as this provides a major advantage over the diffraction efficiency of a grating that peaks around 50 percent at just one wavelength and would require a second order filter to cover the same spectral range. With an HSIM system, the bandpass is the full width half maximum of a narrow emission line and corresponds physically to the width of the entrance slit on the spectrum detector, multiplied by the dispersion at each wavelength. A grating delivers close to linear wavelength dispersion compared to a prism or an electronic tunable filter. The nonlinear dispersion of a prism delivers the lowest bandpass in the blue and highest bandpass in the red.

Figure 3. Spectral segmentation of two user-targeted spectral objects is demonstrated here (a). This graph shows an example of the spectral histogram of aqua and red pseudo-colored spectra, with a ratio of 8.34 to 1, based on 10,009 correlated spectral data points in the field of view (b).

The wavelength dispersion of a system prism is 40 nm at 435.8 nm; for an entrance slit width of 0.025 mm, the spectral bandpass will be 1 nm (40 × 0.025) and changes by approximately a factor of 5 — from 365 to 920 nm. S/N ratio is a function of the efficiency of the prism or grating, the width of the entrance slit, and bandpass. The quantum efficiency of a camera decreases rapidly after about 650 nm, reducing the signal. However, the increase in bandpass of a prism over the same wavelength range boosts that signal, providing an increase in S/N ratio even as the efficiency of the camera falls. Consequently, a prism provides significantly greater effective wavelength range than is possible with a grating. Such a system can accommodate a 25-µm-wide entrance slit, with optimum spatial resolution, S/N, bandpass and camera options.

Spatial resolution

Spectral objects from the FOV distribute along the height of the entrance slit. Spatial resolution — the width of the entrance slit divided by the magnification of the microscope objective — decreases in proportion if camera pixels overfill the entrance slit. The 25-µm slit, as well as an objective magnification of 20×, can deliver a spatial resolution of 1.25 µm. The hyperspectral image included in Figure 1 shows an area of 0.25 × 0.25 mm that has been characterized with 34,669 areas of 1.25 × 1.25 µm. Each pseudo-color represents a spectrum that correlates with a reference spectrum in the RSL.

Cosmetics are an emerging, unique application for HSIM systems.

Object detection vs. resolution

Nanoparticles can be observed in dark-field scattered light even though their size may be below the wavelengths of the light illuminating them. Many nanoparticle fields are akin to stars in the sky, exhibiting unique spectral characteristics that are a function of size, shape and resonance.

Bright point objects present a Gaussian distribution of light that requires a minimum of two, but preferably three, camera pixels in each axis to resolve two adjacent objects. The centroid of the middle pixel indicates the location of a particle in the FOV, limited by the magnification of the microscope objective and the dimensions of a camera pixel. Within these limits, it is possible to count particles as a function of their spectra and map their distribution in the FOV. Particles that cluster, bind or aggregate can be detected along with their distribution in the sample.

HSIM offers a powerful, nondestructive means to perform spectroscopic analysis on heterogeneous materials with micron and submicron spatial resolution. This applies to all industries that combine materials to create an end product, especially when those materials encompass components — such as particles or nanoparticles — that are too small to be visualized in optical wavelengths. HSIM enables these particles to be exquisitely detected within an area as a function of the analytical spectroscopic properties.

Meet the author

Jeremy Lerner is the founder of LightForm Inc. in Asheville, N.C.; email: jlerner@lightforminc.com.

Acknowledgment

Paint data sets are courtesy of the National Center for Forensic Science at the University of Canberra in Australia.