Faster, more compact, and cheaper hyperspectral cameras are making the technology a more appealing item on the menu of food inspection solutions.
HANK HOGAN, CONTRIBUTING EDITOR
If the old maxim “you are what you eat” has any truth to it, then it becomes critically important to ensure that the food you consume is safe and of suitable quality. To this end, the U.S. Department of Agriculture (USDA) spent $1.1 billion
in 2019 to support food safety and inspection operations, while state governments and industrial producers funded additional food safety and quality assurance efforts. Despite this, the Centers for Disease Control and Prevention (CDC) estimates that 48 million Americans get sick every year from food-borne pathogens, foreign objects found in food, or poor-quality product. As a result, industry and researchers continue to seek new ways to improve the reliability, speed, and cost of food inspection.
Unlike the image of meat captured in the visible spectral range (left), the hyperspectral image reveals regions of interest — such as areas of fat or protein, highlighted in false color — to food quality inspectors (right). Courtesy of VISRATEK.
Enter hyperspectral imaging, which combines spectral and spatial data to provide insights about food quality and composition that would otherwise be invisible to the naked eye. Hyperspectral cameras compile spectral and spatial information in the form of a datacube, which can be thought of like a deck of cards. All of the cards share the same 2D spatial image of the light reflecting off an object, but individual cards in the deck disclose a different wave band of reflected light. In other words, the light striking each pixel of a hyperspectral camera is broken down into its component spectral bands. Although there are drawbacks concerning imaging speed and cost, the technology brings clear benefits to food safety and quality checks.
Seeing the unseen
“Hyperspectral imaging enables chemical analysis of products directly on the processing line in a nondestructive way,” said Fatih Ömrüuzun, managing director at VISRATEK in Ankara, Turkey. The systems integrator offers products that combine robotic sorting with hyperspectral quality analysis for food processing and other applications.
Ömrüuzun said conventional food inspection operations typically involve 10 or more human operators at some point visually examining product as it moves past them on a conveyor. The inspectors remove suspect food and foreign objects on the spot but are unable, for example,
to perform quality checks on water or sugar content in produce. This level of
inspection requires samples to be removed from the line and examined in
a lab, which can take hours or days.
In contrast, a robotic sorting system equipped with conventional cameras can perform visual inspections at a higher
accuracy and without getting tired or
sick like people do, Ömrüuzun said. With the addition of hyperspectral technology, the automated system can also sort product into various grades based on water content, sweetness, or other factors.
VISRATEK uses hyperspectral imagers from Headwall Photonics in its
products. Alon Vaisman, a product
manager at Headwall, said that some
hyperspectral imagers may gather data from 100 or more spectral bands, each measuring 5 nm in width. Contrast this
to the dozen or fewer bands, each measuring about 20 nm wide, that are tracked in multispectral imaging configurations. Spectroscopy, which also finds application in food inspection and authentication, only provides information about a narrow point, while hyperspectral imaging captures a full spectral map of the scanned object. “Hyperspectral imaging is spectroscopy and imaging married together,” Vaisman said.
Most hyperspectral imagers operate
within two established ranges: one running from 400 to 1000 nm and the other from 900 to 1700 nm. The first region uses silicon sensors, while the second relies on indium gallium arsenide (InGaAs) technology. Hyperspectral imaging in the shorter-wavelength region discloses subtle changes in color, as well as chemical information, while imaging in the shortwave IR (SWIR) yields more data about the chemical composition of an object, including moisture content. The capability to measure moisture levels is valuable in food quality inspection. Determining the amount of moisture in the lettuce of a salad, for instance, is one way to tell, from a distance, whether the produce is fresh.
Hyperspectral imaging enables vendors and producers of almonds and other foods to rapidly grade product quality in-line without resorting to time-consuming off-line lab analysis. Courtesy of Headwall Photonics.
Hyperspectral imaging can detect blemishes in food products — and other defects that would otherwise be invisible to the naked eye — such as an apple bruise, shown in false color (left). The data analysis is based on the visible-NIR spectral response (top right). Courtesy of HinaLea Imaging.
The full spectral breadth of hyperspectral imaging technology need not be applied to every application. But instances in which a required detection window straddles various sensor and optics solutions could pose a challenge. “Essentially you need different cameras for different ranges,” said Barry McDonogh, general manager for HinaLea Imaging. “You’ll need different optics and sensors for the UV and for the visible/near-infrared — which are typically silicon-based. Then into the SWIR, you need an InGaAs sensor.”
Sony is challenging this limitation with development of sensor technology that covers the spectral range from the visible to SWIR regions.
Spectrum of challenges
According to Vaisman, one potential
drawback to the level of detail that hyperspectral imaging captures is the sheer amount of data generated, which can run
in the hundreds of megabytes per second and present transmission and storage challenges. The data can also be difficult for a nonexpert to interpret or use. Advancements in processors have now made it possible to perform analysis and classification of the imaging data on the hyperspectral device itself, which VISRATEK’s Ömrüuzun said has been key.
“These systems perform classification internally and stream classification
results in various formats, such as a pseudo RGB image,” he said. “So, non-expert users can interpret streamed data.”
Hyperspectral microscope imaging analysis can spot and identify food-borne pathogens. A hyperspectral microscope image of Salmonella typhimurium,
the bacterial pathogen responsible for Salmonella outbreaks. A hyperspectral image of the bacteria (a), highlighted regions of interest (b), and spectral signatures of E. coli, Listeria, Staphylococcus, and Salmonella (c). Courtesy of Bosoon Park/UDSA Agricultural Research Service.
A second prospective drawback to
hyperspectral imaging is its cost. But Vaisman said that this factor must be
balanced against the possible savings. Avoiding a costly product recall, for instance, or being able to rapidly grade or authenticate an item on the line, may make hyperspectral imaging worth the initial expense.
The cost of hyperspectral solutions has dropped by more than two-thirds over the last decade, said Markus Burgstaller, CEO of Perception Park, based in Graz, Austria. Where systems may have run €60,000 to €80,000 ($79,200 to $97,000) in 2012, their cost is dropping below €20,000 ($24,000) today, he said.
Perception Park produces hyperspectral imaging software focused on industrial applications and chemical quantification of materials. The programs, said Burgstaller, can help to sort blueberries by their sweetness, for example, while also enabling detection of foreign contaminants, such as a blueberry-size bug.
While cost is becoming less of a factor for hyperspectral imaging, Burgstaller cited illumination and image resolution as additional challenges, particularly when imaging in the SWIR.
Some industrial inspections in the SWIR range may require thousands of watts of illumination. While halogen lights are a low-cost broadband option for this range, SWIR LEDs are gaining appeal because they emit less heat than halogen bulbs, a feature that is especially beneficial in food inspection applications.
With regard to resolution, hyperspectral sensors operating in the SWIR range have ~640 pixels on a side. In contrast, silicon CMOS sensors have 10× that number. This lower resolution affects some food quality inspection tasks.
“It is hard to do rice sorting or cereal sorting because the [line] speed is high, and the object is small,” Burgstaller said. “It is really challenging to do this with hyperspectral imaging because of the small resolution of the cameras.”
Emerging developments
Researchers at the USDA’s National Poultry Research Center are exploring use of hyperspectral imaging to identify pathogens on chickens. Current methods involve swabbing birds and then culturing the swab to see which, if any, pathogens are present. The process takes days and needs a pathogen load in the thousands. The hyperspectral approach being explored by Bosoon Park, a lead scientist with the USDA, achieves almost immediate results and is much more sensitive.
The approach also couples the technology with a microscope configuration. “We’re looking at a single cell,” Park said.
Pathogens are identified by examining visible and/or near-IR spectral signatures, which differ by pathogen species, and the technique could be useful in other areas of food processing. The research is proceeding with support from HinaLea Imaging, which is supplying imaging hardware and targeting algorithms.
Challenges include processing the high volumes of data produced, Park said, as well as developing calibration strategies and turnkey systems so that nonexperts can produce repeatable results. HinaLea’s McDonogh echoed the sentiment by noting that, in addition to becoming less expensive and more compact, hyperspectral systems for food production must also become easier to use. Hyperspectral devices have also traditionally been relatively large, which makes them difficult to mount on a boom as is possible with a machine vision camera.
McDonogh sees hyperspectral imaging as competing with traditional machine vision, to a degree, as well as augmenting it. The fundamental difference between color RGB systems and hyperspectral imaging technology, he said, is the latter’s spectral resolution and how the additional information is used.
HinaLea’s cameras employ an area-scan rather than a line-scan configuration. Thus, its technology captures hyperspectral information for an entire area at once without the need to scan across it. The result is smaller and less expensive systems with improved spatial uniformity, McDonogh said.
Cubert, a company based in Ulm,
Germany, offers what it describes as a full-frame or snapshot hyperspectral imaging system. Currently, the technology targets the 350- to 1000-nm spectral region, though Cubert is looking into extending the spectral coverage of its cameras.
András Jung, Cubert’s co-founder,
said the company’s products deliver a hyperspectral datacube within milliseconds — a real-time snapshot capability. “Snapshot cameras use no moving parts and significantly reduce the integration time in comparison to conventional line-scan-based hyperspectral imagers,” he said.
Despite the interest in SWIR imaging, plenty of food inspection applications benefit from hyperspectral imagers that operate between the visible and NIR. Distinguishing fat, protein, and muscle in meat, for instance, is possible to a limited degree using the near-IR. But VIS-IR
systems such as Cubert’s snapshot technology perform well at tracking the mixing of liquids or the dissolving of solids into a liquid solution.
When capturing hyperspectral images across very broad spectral ranges, such as from 350 to 1700 nm, a trade-off is made between spectral channels and spatial resolution. As the number of channels increases, the resolution worsens if the number of pixels remains fixed. So, system builders such as Cubert must find a suitable balance between spectral and spatial performance.
Future spectrum
The consensus among hyperspectral imaging vendors is that the cost of the technology will continue to drop even as its capabilities rise. Among the expected developments is the use of machine learning to help systems distinguish various grades of product from one another.
These improvements could lead to the increasing use of hyperspectral imaging because it makes more economic sense, said VISRATEK’s Ömrüuzun. “Considering the increasing demand for having a limited number of employees for quality inspection, [hyperspectral imaging]-enabled industrial systems will be more popular than ever before.”