Line-scan cameras capture images that are one pixel long and thousands wide. Recent innovations involve capturing and merging multiple high-resolution images, even in low light.
HANK HOGAN, CONTRIBUTING EDITOR
When Artemis Vision took on a label inspection application, the client had a set of stringent conditions: They required high-resolution image capture and lighting, along with the ability to manage high throughput. They also needed to acquire the
images even when label dimensions changed.
Courtesy of iStock.com/visual7.
“We needed uniform images across variable length and width fields of view,” said Tom Brennan, president of Artemis Vision.
Variations in illumination could lead to misreading a label. The solution was to get up close — by using a contact image sensor, a line-scan system that places the camera and lighting millimeters away from the label.
“Contact line scan achieves great uniformity because all the pixels have effectively the same optics,” Brennan said.
In addition to inspecting labels, line-scan cameras can also capture foreign objects in food, ensure that flat panels are free of defects, and help to speed
up DNA analysis. Innovations in the technology include the capability to capture and merge multiple images to achieve high resolution in low light, along with incorporating multiple
visible and nonvisible spectral bands to enhance vision capabilities. Software enhancements and other changes have also made line-scan cameras easier to use.
A label inspection system with a contact image line-scan sensor — an integrated and compact solution that works best when space is tight and inspection surfaces are flat. Courtesy of Artemis Vision.
Line-scan cameras now offer depth information, enabling problems on roads, such as potholes, to be detected from a moving vehicle. Courtesy of Chromasens.
As their name implies, line-scan cameras capture images that are one pixel long and up to thousands of pixels wide. Objects move past the imaging line, enabling vision systems to acquire valuable information. This information can then be used to activate tasks such as directing an air jet to flick out small rocks, bugs, and other unwanted items from a bed of rice, for example.
Contact image sensors, the type of line-scan technology used by Artemis
Vision, have been around for some time. They may have a working distance as short as 10 mm, said Mike Grodzki, a line-scan product manager at Teledyne DALSA. Contact image sensor systems are highly integrated and very compact, and they are therefore useful when space is at a premium.
In the visible spectrum, rocks are hard to distinguish from coffee beans (top). Imaging with a line-scan camera in the SWIR makes the distinction clear (bottom). Courtesy of Hamamatsu.
A measurement comparison of four types of plastics. In the visible, the different types appear the same (top). Imaging in the SWIR (center) makes it possible to distinguish one type of plastic from another (bottom), providing vital information in recycling processes. Courtesy of Hamamatsu.
They have constraints: The surface must be flat. And magnification is 1:1, which means that features smaller than a pixel may not resolve.
In cases where contact image sensors are not the solution, other line-scan technologies may be, especially in inspections that require examination of very fine details. This is the case when inspecting lithium battery electrodes. Inspection is often performed by a line-scan camera that stands off at some distance. Remote scans require precise alignment between light sources, optics, and the camera, along with tweaking of the camera and illumination parameters.
Because line-scan cameras may only have an exposure time of microseconds to image a single line as objects move past, the cameras may be photon starved. One approach is to increase the lighting intensity. In some situations the material cannot tolerate intense light, and the resulting heat can cause damage, such as when sequencing DNA samples.
Line-scan cameras inspect food and other products for defects. Courtesy of JAI.
Multispectral imaging is increasingly used to detect problems that are invisible to the eye. Courtesy of JAI.
Collecting more photons
One solution is time delay integration (TDI) line-scan technology. This approach merges synchronized, short, sequential exposures from multiple rows into a single image. TDI can achieve a higher resolution, without changing optics or lighting, when systems use pixels that are offset from one another by half a pixel. Thus, the technology collects more photons and improves the signal-to-noise ratio when photons are few.
“It’s basically increasing the effective exposure time, and then we can reduce the light intensity,” Grodzki said. “For example, 100 stages TDI is equivalent to 100× the exposure time.”
TDI technology does require careful synchronization between imaging and object speed. For some applications, this approach solves significant problems, as in the case of gene sequencing, given the throughput requirements and the required sensitivity and resolution.
Another area of expanding capacity involves innovations aimed at extracting additional information. Sometimes this gain in capacity comes in the form of depth or 3D data, with one example being line-scan cameras that image a road surface from a moving vehicle.
At other times, the additional data involves other parts of the spectrum, such as shortwave IR (SWIR), which runs from just above the visible and near-IR at about 1000 to 1700 nm, or even — in some implementations — 2500 nm. Teledyne DALSA is seeing increased interest in SWIR because this wavelength range can pick out details below the surfaces of fruits and other foods. It can also extract information such as water content, which is important for gauging food quality.
Other companies also see SWIR line-scan technology as a growth area due to application demand. Chromasens
announced a new SWIR line-scan
camera and illumination system in November 2021.
“SWIR can see beyond the visible spectrum, such as detecting moisture levels, finding impurities in seeds, or monitoring for impurities in pharmaceuticals. Materials that look identical to the human eye can be made visible in the SWIR spectrum in different ways, since all materials absorb light in different ways,” said Klaus Riemer, Chromasens’ product manager.
Critical to the use of a SWIR or any other type of line-scan camera is an appropriate light source that matches the sensor’s spectral sensitivity and causes defects or items of interest to appear differently. The illumination must also be bright enough. Riemer said a 200-kHz scan rate means that the system images each line of pixels in 5 ms. Such a short exposure time may mean that illumination levels need to be over 1 million lux — about 9× as intense as bright sunlight.
SWIR line-scan cameras are often used in food sorting, said Albert Tu, product marketing manager for imaging solutions at Hamamatsu. For example, small rocks may be difficult to distinguish from coffee beans in the visible range. In the SWIR range, however, the rocks look clearly different than the beans.
Imaging in the visible is typically done using CMOS sensors, which capture an object as it appears to the eye. In contrast, imaging in the SWIR range, which is done using indium gallium arsenide (InGaAs) sensors, highlights water
content and therefore internal bruising
in an apple, for example, or other
defects that are invisible to the eye.
Line-scan cameras that are sensitive out to 2200 nm can distinguish between different plastics that look identical to the eye. “You can identify these materials throughout the recycling process and separate it,” Tu said.
Lithium battery electrode inspection can include line-scan cameras that look for defects. Courtesy of Teledyne DALSA.
A diagram of a typical setup of line-scan cameras arranged to perform lithium battery electrode defect detection. Courtesy of Teledyne DALSA.
This ability is critical because different types of plastic require different recycling processes. Without SWIR-sensitive line-scan cameras, or another nonvisible imaging method, sorting must be performed by hand — an expensive and slow process.
Combining spectral bands
A final hardware innovation combines these various spectral bands into one system in which three sensors acquire the primary colors of red, green, and blue while a fourth captures monochrome, near-IR, or SWIR data. Using a prism and filters to achieve this merging of sensors offers advantages, according to Paritosh Prayagi, director of product management at JAI.
Important benefits of this method are that the optical paths for the various spectral regions align with one another and the different spectral bands have the same magnification. This optical matching leads to crisper images when inspecting wavy or nonflat objects than is possible with a competing approach using a multiline sensor, Prayagi said.
A prism-based line-scan camera can also better handle
instances in which a conveyor belt or other material-movement web mechanism is vibrating or objects on it
are moving. Either one or both of these conditions can
happen when transporting food from one location to the next, for example.
Prayagi said of the prism-based line-scan technology, “It can manage this nonuniform motion quite well.”
Users are increasingly turning to these multispectral band solutions, he said, and industry is responding. Today, JAI’s multiband solutions may cover three or four spectral regions, ranging from 400 to 1000 nm. However, the company will soon go to solutions that cover the spectrum to 1700 nm.
A prism-based solution is limited by how large the prism can be, which is set by the cost of the optical element. This limit, in turn, sets a maximum feasible resolution, Prayagi said. According to Hamamatsu’s Tu, trade-offs always exist. Sensitivity affects speed or resolution, spectral response range interacts with sensitivity and sensor technology, and cost is always a factor.
At one time, line-scan cameras had a reputation for being cumbersome and hard to work with. Software improvements have made them easier to use, and vendors are working to make operation even simpler, Teledyne DALSA’s Grodzki said.
Despite all these advancements and innovations in line-scan cameras, Artemis Vision’s Brennan said that, for some applications, an area-scan camera may be a better choice. The falling prices, increasing pixel counts, and rising speeds of area-scan cameras make them the preferred option for a wider range of applications than was the case before.
Sometimes, however, the application calls for inspecting a huge swath of material in fine detail, which means that, for imaging purposes, the area of interest is tens of thousands of pixels wide and arbitrarily long. The material may be paper, film, or rice that is spread across a conveyor belt and tumbling into a bin.
In such circumstances, Brennan said, there is only one choice in vision technology. “I think that application will always be line scan.”