David Bannon and Christopher Van Veen, Headwall Photonics Inc.
A technology that evolved from the cloaked secrecy of military satellites and reconnaissance technology is finding its way into poultry- and produce-processing facilities.
Hyperspectral imaging holds so much promise across so many applications, and companies specializing in this technology are working to drive out complexity and cost, which will make it a more mainstream inspection technology for improving food safety and quality.
(a) Spectral analysis of specialty fruit identifies contaminants: wood (blue), metal (green) and disease (orange). (b) In-line process inspection of contaminated, diseased fruit. (c) Visible and NIR (400 to 1000 nm) analysis finds bruised and unbruised strawberries.
Three primary areas of food and agriculture have adopted hyperspectral imaging. The first is in-line processing and inspection of everything from specialty (high value) crops such as strawberries and apples to meat products such as poultry and seafood. The second is remote sensing of agricultural areas; with this technique, data-rich analysis of crops and farmlands can be performed from an airborne platform. The third is an emerging category of plant research and crop science incorporating plant phenotyping and genomics. Here, hyperspectral imaging can assist research efforts to analyze gene functions that become key elements in plant breeding, with the goal of optimizing growth and disease protection. In this market, advanced imaging technology enables a new class of agricultural research and in-line, high-throughput inspection of food products.
Not just for the lab
Hyperspectral sensing is not a “new” technology, per se – it has been around for about 20 years. Long favored in defense and reconnaissance circles, it was seen as complex and dedicated to airborne applications. These characterizations were not entirely untrue, but over time, the ability to see and classify objects based on the inherent chemical composition or spectral signature became a value proposition that could not be ignored by all sorts of commercial endeavors, especially the inspection of foods on high-volume processing lines.
Green apples whiz by on an inspection line. Images courtesy of Headwall Photonics Inc.
From a regulatory perspective, very few industries depend on rigorous inspection more than food processing. Governmental oversight is more rigid than ever, which means that these companies cannot simply operate with systems and processes to their liking. Disease, contaminants and other health-related conditions must be identified quickly and completely, whether a company is processing specialty crops, seafood or poultry. Passing muster means labor-intensive inspection processes, sometimes on a continual basis and across multiple facilities. Under such tight regulation, private-sector companies feel continual pressure to somehow remain efficient and profitable.
Because of this pressure, food-processing companies invest in spectral imaging instruments to make the inspection side of their operation more than pay for itself. After all, rejected products “cost” much more the later they are found. Occasionally, contaminated products do reach store shelves; the financial cost, coupled with the public relations backlash, can cripple a brand for years or even result in closure of operations.
Disease conditions, contaminants and other health-related concerns
must be identified quickly and completely, whether a company is
processing poultry, seafood or specialty crops.
One way for food-processing companies to invest in new technologies while seeing a demonstrable return on investment is to make sure the inspection processes are rapid, accurate and repeatable. High throughput is key, because the faster a facility can inspect its products, the more efficient the operation becomes.
Any increase in speed must not diminish safety or quality, however, so the US Department of Agriculture (USDA) is thoroughly investigating new ideas and technologies in limited pilot programs across the country. For example, the department is planning to fundamentally change the way poultry products make their way to the American dinner table. If new automated technologies can help reduce manual labor costs while quickening the inspection process, consumers benefit from food products that not only are safer but also more affordable.
The USDA is investigating new technologies to fundamentally change the way poultry products make their way to the table. Automated technologies can reduce labor costs and speed inspection, resulting in healthier, more affordable food products.
Hyperspectral sensing has an advantage in food inspection because it can see things that other machine-vision technologies cannot. Because every object has its own unique spectral signature, a hyperspectral sensor can noninvasively and rapidly scan products for anomalies. Fecal and chemical contamination, disease conditions and foreign objects can be “red flagged” once these triggers have been properly cataloged with the right spectral signature. Equipped with these unique “spectral libraries,” the sensors go about their business of determining good product from bad.
Spectral bands of interest comprise everything from UV (250 to 600 nm) all the way up to SWIR (950 to 2500 nm), and include VIS (380 to 825 nm), visible and NIR (380 to 1000 nm), and NIR (900 to 1700 nm). A poultry-processing facility, for example, may have a very distinct set of anomalies for which it checks. Each condition will therefore fall into one of these spectral bands, and the hyperspectral instrument will look for these.
Although the technology behind hyperspectral sensing is generally well understood, adapting it to the rigors of hot, wet, noisy and unappealing conditions is a challenge requiring experience with harsh environments. First, the imaging systems must be reliable and affordable. Instrument reliability is a factor derived from the imaging performance as well as the spectral and spatial resolution of the hyperspectral instrument.
Here, the use of aberration-corrected diffractive optics – representing the heartbeat of the instrument – is a distinguishing characteristic. Headwall Photonics, for example, manufactures in-line hyperspectral sensors that are corrected for aberrations. This patented design allows the sensor to inspect a scene across a very wide process conveyor line without image distortions that could affect the measurement. Aberration correction also yields higher signal-to-noise ratios, and levels of spectral and spatial resolution. In practical terms, a precisely engineered grating leads to a precisely engineered hyperspectral sensor; a food inspection line, for example, will see fewer false-positives if the spectrometer uses aberration-corrected gratings.
How in-line high-speed hyperspectral sensing works: (a) As the product moves under the sensor, a visual image of the sample is rendered by unique signatures; (b) the full spectra are captured for any position in the field of view; and (c) the visual image of the sample at any wavelength is captured.
To gain full processing line coverage, a wide field of view is important in a hyperspectral sensor. These systems are built upon the push-broom architecture, with food products passing by the hyperspectral inspector. For each moment in time (each frame capture by the sensor), the scene observed by the fore lens is imaged through a tall slit aperture within the instrument. The scene that fills this slit aperture is then dispersed through the spectrometer (containing the grating), with the resulting spectral and spatial information imaged onto a two-dimensional focal plane array (FPA) using silicon CCD, InGaAs or HgCdTe arrays. One axis of the FPA (pixel rows) corresponds to the imaged spatial positions within the field of view all along the slit height. The slit height determines the overall field of view. The second axis (spectral for pixel columns) corresponds to the spectral wavelength that is linearly dispersed and calibrated. Each 2-D image (or frame capture) is digitized by the FPA to build a data set – representing the hyperspectral scan – that comprises all the spectral and spatial information within the scene or field of view of the sensor.
The user can evaluate any point or pixel within the field of view for its chemical spectral signature while maintaining the integrity of the spatial information obtained. Hyperspectral sensors also can interrogate spectral signatures of interest, based on the defined algorithms. When these algorithms are used on the processing line, “accept” or “reject” decisions can be passed to robotic arms or air knives for product removal. Headwall Photonics has worked very closely with USDA researchers to establish algorithms and spectral libraries for a wide number of agricultural products.
Sensing from above
One interesting use of hyperspectral sensors is in precision agriculture, where small, lightweight sensors are deployed aboard airborne platforms ranging from Cessna aircraft to unattended aerial vehicles.
In the same way that food-processing lines are evolving from straightforward machine-vision systems to hyperspectral sensors, many growers and agriculturists also are moving to hyperspectral imaging. The richness of the data collected can give farmers a sense of what and where to plant, and when to fertilize and harvest. High-value crops such as pecans, grapes and walnuts must be managed with precision. Nutrient levels, ripeness and disease can be “seen” by hyperspectral sensors in much the same way that food-processing lines can be seen. Because these sensors are deployed aboard aircraft, hundreds of acres can be surveyed quickly – and the data-processing power coupled to these sensors means that more useful information can be obtained. The result is better overall crop management across the globe.
VineView Scientific Aerial Imaging of St. Helena, Calif., uses high-resolution aerial imaging and scientifically calibrated data products to assist in crop uniformity optimization, irrigation management and harvest planning. VineView has used infrared sensors for the most part, but the company now is going beyond IR to hyperspectral.
“Adding hyperspectral data allows us to provide more specific actionable information to our clients who manage high-value crops,” said Dr. Matthew Staid, the company’s president. The hyperspectral sensing technology means that VineView not only can map vigor or stress within crops, but also can better identify the specific causes of those stresses. The presence of leaf roll, weeds and invasive species will be mapped as VineView aircraft patrol orchards and crop fields.
The Hyperspec data processing unit from Headwall Photonics analyzes spectral and spatial imaging data cubes, which can grow to several gigabytes in size; extremely rapid data acquisition and analysis are needed to extract spectral features of interest.
At the Instituto de Agricultura Sostenible in Córdoba, Spain, small, unmanned aircraft outfitted with high-resolution sensors look down on crop fields and orchards to make precision planting and harvesting decisions. Because the sensors weigh less than 2 lb and exhibit excellent spatial and spectral resolution, they are perfectly suited to giving farmers and agriculturists a wealth of previously “unseen” spectral information from altitudes of 1000 to 15,000 ft.
Although the sensor technology is there to see what is on the ground, hyperspectral data processing can be processor-intensive. Fortunately, data-processing horsepower is similarly increasing in capability while costs continue to go down or at least moderate. Headwall Photonics, for example, offers its Hyperspec data processing unit, which focuses on the analysis and interrogation of the spectral and spatial imaging data cubes. Hyperspectral data cubes can grow to several gigabytes in size, and they require extremely rapid data acquisition and analysis to extract spectral features of interest. They also depend on specific algorithms and the classification or search for certain threshold conditions within the scene.
Plant phenotyping, crop science
The ability of hyperspectral sensing to see beyond the visible also makes it a valuable tool in plant phenotyping and crop science research. In simple terms, a phenotype is a composite of an organism’s observable characteristics or traits. These can include its morphology, development, biochemical or physiological properties, phenology and products of behavior.
The practical application of phenotyping is to assist in plant breeding: identifying traits such as disease resistance, enhancing growth and development characteristics, and more. Seed companies, for example, can use this science to introduce higher-quality, hardier specimens for certain environments and regions, and to standardize growth conditions more accurately than ever.
Würselen, Germany-based LemnaTec GmbH has been a pioneer in the science of phenotyping since 1998. The company uses Headwall’s Hyperspec Inspector in its harsh “greenhouse” environments to analyze the chemical composition, overall structure and specific traits of plants passing within the field of view. Hyperspec Inspector is the same type of “point and stare” system that food-processing facilities use, but it is optimized for specific spatial and spectral resolutions.
The advantage of hyperspectral imaging systems is that, although the basic fundamentals are the same from one system to the next, key parameters such as spectral bands, resolution, signal to noise and lighting conditions can be modified to suit each specific application. And because these systems are deployed in environments where humidity, moisture, vibration, heat and chemicals are often the norm rather than the exception, the enclosures are very protective of the instruments contained within.
Meet the authors
David Bannon is CEO of Headwall Photonics Inc.; email: [email protected]. Christopher Van Veen is in marketing at Headwall; email: [email protected].