Dr. Leah Ziph-Schatzberg, Corning Specialty Materials
Hyperspectral imaging systems supply classification and identification information, in addition to image information. This property makes them useful for numerous industrial and other applications, many of which have yet to be discovered. New technology and manufacturing capabilities make it possible to design and fabricate systems that are compact, high-performing and cost-effective. With these new systems, the market need for hyperspectral information will grow rapidly, as awareness and accessibility increase.
Hyperspectral systems manufacturing that is vertically integrated – where design, analysis and production
of slits, gratings and all the optical and mechanical components can take place under the same roof – is well-positioned to meet this growing market need. Vertical integration enables system designs that snap together, require few parts or alignments, and are minimal in size, weight and cost.
Hyperspectral imaging (HSI) is a combination of spectroscopy and imaging, so it is often also called imaging spectroscopy. This imaging modality produces a data cube (or hypercube) comprising 2-D spatial information and a third, spectral dimension, providing a full spectrum for each pixel in the imaged scene. The spectral information identifies the materials in the scene, while the spatial information provides location.
Figure 1. A hyperspectral data cube (hypercube). This image was acquired in 1997 over Moffett Field in California by the Airborne Visible-Infrared Imaging Spectrometer, AVIRIS, developed by NASA’s JPL Laboratory. Photo courtesy of NASA.
Hyperspectral imaging can gather spectral and spatial information in wavelength ranges from the UV to long-wave IR. The choice of wavelength range and imaging sensor (focal plane array) is dictated by the application. Targets of interest have spectral signatures in various wavelength ranges, prompting the choice of the appropriate hyperspectral camera.
HSI systems can work in transmission, reflection and emission (e.g., fluorescence), depending on the configuration of the illumination and the position of the camera relative to the sample. The HSI system itself is typically agnostic to the configuration and measures the light impinging on it. The characteristics of the sample itself can be determined by calibrating the system for reflection or transmission, so that the spectral signature of the illumination is removed.
Common terminology for spectral imaging includes the words “multispectral” and “hyperspectral.” Multispectral imaging typically refers to systems where a small number of wavelength bands are recorded. They can be contiguous, overlapping or noncontiguous. Hyperspectral typically refers to a large number of contiguous wavelength bands, allowing the spectrum to be collected at high resolution. Multispectral systems are potentially faster in accumulating and processing information, and may be useful for some applications. In situations where the targets are unknown or varying, or made up of complex mixtures, hyperspectral is the required tool. A hyperspectral system is more versatile and flexible, because the bands of interest can be modified via software for a certain application, taking advantage of more efficient data collection and processing. The same system can then be modified for another application or target of interest.
HSI system modes of operation
A few ways exist to collect hyperspectral data: push-broom, scanning the wavelength (e.g., with tunable filters), snapshot, image slicing and Fourier transform infrared (FTIR).
A pushbroom hyperspectral system – the most commonly used for remote sensing, intelligence, surveillance and reconnaissance – is shown schematically in Figure 2.
Figure 2. Typical pushbroom optics. From F. Dell’Endice et al (2009). Improving radiometry of imaging spectrometers by using programmable spectral regions of interest. ISPRS J Photogramm Remote Sens, Vol. 64, Issue 6, pp. 632-639.
A line in the scene is imaged onto a slit entrance to a spectrometer. The spectrometer includes a dispersive element, splitting the collected light into its components. The spectrometer images the dispersed line image onto a focal plane array (FPA), where one axis records the spatial information (along the slit) and the other axis records the spectral information (perpendicular to the slit). This 2-D frame represents spectral and spatial information from one line in the scene. Moving to the next line, the system collects data in the next frame. Repeating this process, the system collects a data cube.
This mode of operation is ideal for situations where the camera and the scene are in relative motion, as occurs in applications such as in-process monitoring. An example is a conveyor belt moving a sample(s) in front of a stationary camera. Also suited for this mode of imaging are airborne applications, where the camera is flying over a target. The high-quality dispersive elements such as diffraction gratings typical to this HSI architecture provide high spectral resolution and fidelity.
Another way of building a data cube is by scanning the wavelengths. Each frame of the FPA records the scene at a single wavelength, and the next frame records the scene at another wavelength, continuing to build the data cube one wavelength band at a time. In this case, each frame contains full spatial information and partial spectral information from the scene. These systems may include tunable filter based architectures or digital light processing chips. This mode of operation lends itself to situations where the spectral content will not change during the wavelength-scanning time, with stable illumination conditions and a static sample. Liquid crystal tunable filter systems typically require long integration times to compensate for the inherently low efficiency of the LCD filter. Acousto-optical tunable filters typically have higher efficiency and spectral resolution than liquid crystal tunable filters. They are, however, bulkier and require power to operate, in contrast to the passive diffraction gratings in pushbroom systems.
Figure 3. This compact visible and NIR system from Corning enables industrial and other applications of hyperspectral imaging.
A true snapshot-hyperspectral system divides the FPA so that each cluster of pixels is coated with different dichroic filters and represents a single spatial “pixel.” Such systems typically compromise the spatial and spectral resolution because they divide the FPA into areas, collecting both spatial and spectral information in a single frame. These systems also use a small portion of the available illumination because they filter out most of the information, whereas a pushbroom system collects it all. Another disadvantage is the angular dependency of the filter transmission. This inherent property results in the filters transmitting different wavelengths, depending on the angular content of the light. The outcome is lower spectral fidelity, typically causing the wider spectral bands to overlap in an uncontrolled manner. These systems are best utilized for high-volume applications requiring lower spatial and spectral resolution and fidelity.
Several image-slicing techniques exist, ranging from lenslet or mirror arrays to fiber optic bundles. These systems typically reformat a 2-D image to a line image, then disperse the light and reconstruct a 2-D image, recording the spatial and spectral information. When operating in snapshot mode, these systems share the same resolution issues of the filter-based snapshot systems. In addition, they are typically larger and more expensive to manufacture.
FTIR hyperspectral systems work as FTIR single-pixel spectrometers but are combined with FPA detectors. Each pixel records the interferogram from a specific location in the scene. The interferogram is a sum of cosines, where each cosine component is created by interference of monochromatic light. Processing the data by Fourier transform at each pixel results in the spectrum for that location. FTIR systems can have high spectral resolution, which makes them useful for applications such as gas analysis. Typically large and expensive, these systems require significant signal processing to create the hypercube. In addition, they have moving parts (a reliability issue, as they are less rugged) and therefore are less attractive for airborne applications.
Applications
Applications for hyperspectral imaging are numerous. The power of full-spectrum information combined with localization of the spectral signatures opens up rich capabilities. Compact pushbroom HSI technology enables applications previously unavailable because of the size, weight, power and cost of systems on the market. These compact systems allow for portable systems and applications where the available space is very limited.
To get useful information from HSI in airborne applications such as oil and gas exploration, precision agriculture and remote sensing (e.g., environmental monitoring), the hyperspectral data must be correlated to locations on the ground. To achieve this, the system must include the sensor, plus additional hardware and processing. The sensor data must be combined and synchronized with flight information, such as heading, speed, inertial information and location. This information can be gathered from a combination of inertial measurement unit and global positioning system (GPS) data. These systems must all be synchronized so that the HSI data can be mapped to a ground location. This process, called georectification, typically involves taking the HSI data in its original geometry and incorporating it into a map projection. Some applications also require orthorectification, which takes into account the topography of the imaged area. The HSI systems targeted at these applications are therefore more complicated than just the sensor itself. However, the complete system with the added components can still be very compact.
Figure 4. The Corning application development kit is delivered with Corning Hyper-C software and a tablet that allows users to develop their own applications.
Airborne applications usually require high spatial and spectral resolutions to maximize data usefulness; they are ideally suited to pushbroom systems. Compact hyperspectral systems allow for installations in small manned and unmanned aerial vehicles, providing high performance in a small, lightweight package.
Ground applications for HSI are less established in the marketplace. A significant emerging application is in-process monitoring. The versatility and utility of HSI technology allow for better sorting, recognition of defects, and identification and elimination of health hazards, offering significant overall improvements to standard machine vision. The added spectral information increases the ability to detect and identify targets.
Other ground-based applications include surveillance for law enforcement agencies and first responders, as well as biomedicine. Multiple groups around the world are working on developing applications in the medical field, especially cancer detection.
Because of their small footprint, compact hyperspectral systems offer a significant advantage for many mobile, in-process monitoring and medical applications. A new turnkey application development kit from Corning allows users to develop applications with ease. Corning Hyper-C software controls the system and provides basic analysis of the collected hypercubes, as well as the ability to download them into other image-processing and analysis applications such as Matlab® and ENVI® for further processing.
Analysis algorithms
The analysis algorithms must be developed specifically for each application, depending on the information required. These algorithms will depend on the task at hand. Some applications require classification among several categories of interest – for example, different vegetation states. Other applications require the detection of a signature of interest in the background comprising multiple spectral signatures. Still others are looking for anomaly detection (e.g., man-made materials in a natural background).
Each task requires a different type of algorithm. A number of common algorithms are being used to gather significant information from collected data cubes. To analyze the data from airborne systems, the data cubes are typically calibrated to measure reflectance by removing lighting and atmospheric effects. Since hyperspectral data is so multidimensional, principal component analysis is frequently used to reduce the dimensionality before commonly used or specifically developed classification/detection algorithms are applied.
In summary, compact, low-cost, high-performing hyperspectral systems enable a vast number of industrial applications that were not possible before they were released.
Meet the author
Dr. Leah Ziph-Schatzberg is a program manager at Corning Specialty Materials in Keene, N.H.; email: [email protected].