Fast and efficient delivery of product into the customer’s hands is the endgame in manufacturing. Any defect that occurs as products progress along the manufacturing line may be magnified, which can cause bottlenecks in production or require expensive fixes down the line. When high volumes and yields are added on top of the goal of quality control, problems must be caught quickly and reliably. Such demands help to explain exactly where and why high-speed imaging has become critical to applications across numerous industries, from packaging to logistics to automotive. High-speed imaging captures defects in an automated glass inspection system. Courtesy of ATS Automation. Factors to consider in the design of high-speed industrial imaging systems include the camera’s trigger latency, field of view, and shutter speed, as well as other vision components such as lighting, cabling, and processing speed. There are system-level considerations as well, such as how field of view influences exposure times. Getting all of these factors right is necessary when freezing fast-moving objects in time while minimizing image blur, and this need plays out in many ways, depending on the environment and the application. Acceleration in automation In manufacturing and industrial settings, high-speed imaging means different things to different people, said Steven Wardell, the director of imaging at ATS Automation. It comes down to the application and the industry, as well as the given needs of a particular setting. The machine vision systems that ATS assembles acquire dozens of images per second. Wardell said that not only the camera but also the lighting, interface, and central processing unit must all be able to accommodate the same flow of data to effectively isolate any defects. “When you’re building components, you’re looking at it in terms of parts per second, and three or four parts per second is pretty typical,” Wardell said. “The vision system has to be able to react to any product flaws that exist. The critical question is the type of inspection — if it’s a simple color code, you can use less critical hardware and you don’t need to worry about blur, for example.” Either xenon lamps or LED strobed or pulsed lighting can help to enhance high-speed imaging more than the actual shutter speed of the camera, he said. But systems integrators have other options too, Wardell said. The limited format of line-scan cameras makes them the tool of choice for inspecting web-based processes and other wide-format products. To help one of its customers grade veneer sheets, Basler configured a line-scan camera and interface card to perform an automated optical inspection of only four lines at 2000 Hz. The system then efficiently stitched the grouped lines together before transferring the complete frames to the host software. Courtesy of Basler AG. “Maybe the whole frame doesn’t need to be analyzed, but [only] certain points,” he said. “A system might be inspecting 10 parts per second, or 100 milliseconds per part. We think of it all the way down the line, in terms of the time it takes to get the image and process it. All the bits and pieces add up. “When we look at technology advancements in high-speed imaging, it all comes down to resolution and speed,” he said. “Twenty years ago, we [may have been] looking at a 600- × 800-pixel image at 20 fps. Now, we might be looking at a 25-MP image at 10 or 20 fps. It’s not just about how fast an image is taken, but how to keep the process continuous.” A receiving tunnel in a distribution center captures high-speed images of the color and size of packages, as well as their label data for sorting and shipping. Courtesy of Cognex Corp. Resolution and speed Many machine vision applications — such as the inspection of flat panel displays, printed circuit boards, and semiconductor wafers — place equally high demands on resolution and speed, which can be difficult to balance. This is why companies such as Teledyne DALSA are developing superresolution techniques based on CMOS sensor-level designs. High-speed TDI cameras, such as those offered by Teledyne DALSA, can run at 400 kHz at 8K, 16K, or 32K resolution, said Xing-Fei He, senior product manager at the company. “What we want is 32K at 2.5-μm resolution, and that is very difficult to achieve.” He added, “An optical lens has an MTF [modulation transfer function] limitation of a pixel size of about 3.5 μm. So we are using two arrays of 5-μm pixel size in our cameras, with a half-pixel offset.” He said that offsetting two 16K, 5-μm arrays by a half pixel allows for the capture of two images at the same time, which can then be reconstructed by the frame grabber to reproduce a 32K/2.5-μm image. The technology is thus dependent not just on the camera itself, but on the frame grabber, which offers benefits for ultrahigh resolution while using the existing lens. At the same time that Teledyne DALSA has been focusing on superresolution R&D, it has also made strides in multifield technology, He said. In the coming months, the company is planning to unveil a camera that comes complete with a CMOS chip featuring three timedelay- integration (TDI) arrays. This will allow different lighting at different wavelengths and angles to be scanned at the same time. “The customer needs to scan three times to capture three different images,” He said. “With this technology, one scan will be able to capture bright-field, darkfield, and backlit images in a single scan, without slowing down throughput.” Answers in logistics High-speed line-scan cameras have long been the workhorse for reading barcodes in logistics applications, helping companies and distribution houses to ship in bulk. But vision capabilities are evolving to address the growing variation in logistics operations, according to Ben Carey, senior manager of Logistics Vision Products at Cognex Corp. Laser-based barcode scanners were once sufficient for the track-and-trace solutions industry, but the current volume and variation of packages traveling through distribution centers requires collection of more data, more quickly. This trend has prompted the need for more advanced vision technology. “With logistics, there could be any number of things you’re looking at, including an envelope, a pad, various designer labels, and a barcode associated with each,” Carey said. “But in the last three to five years, our research and development people have been looking at means to automate logistics.” ‘The customer needs to scan three times to capture three different images. With this technology, one scan will be able to capture bright-field, dark-field, and backlit images in a single scan, without slowing down throughput.’ — Xing-Fei He, senior product manager at Teledyne DALSA Carey further said that, as the COVID-19 pandemic has spread around the world, this growth in data and the need to collect it has only accelerated as more people buy goods online and expect next-day delivery for both small- and large-scale fulfillment of orders. To support the growing volumes and variations in distribution operations, high-speed imaging systems have come to support multiple-camera systems, a broader analysis of pixels, and larger fields of view that image at more acute angles. Cognex, for example, borrowed from smartphone camera technology to develop its own high-dynamic-range imaging system that captures multiple pictures at various exposures to form a composite image. The technology can help capture barcode or other package data that may otherwise be obscured on a crowded, fast-moving conveyor belt. While motion has historically posed problems for the multiple-capture, high-dynamic-range approach, Cognex further developed algorithms to compensate for this drawback by selectively pulling relevant data from the sensor’s bit depth. With the growth in e-commerce, high-speed imaging systems in logistics and distribution centers must be able to scan a variety of packages — regardless of the packages’ size, shape, orientation, or placement — on fast-moving conveyors. Courtesy of Cognex Corp. “Another aspect of this technology is that the focal plane has to change drastically as the process continues,” Carey said. “So Cognex has invested in liquid lens technology that allows imaging whether it is 200 mm or 10 mm away from the camera lens. This allows fewer cameras to be used for the solution.” These and other developments have shifted barcode-reading from laser-based scanners to image-based solutions, which can streamline operations, enhance read rates, and provide other useful information about a package. Where a line-scan system would have simply noted a missing piece of barcode and rejected an item, new vision technology is confirming the presence and placement of barcodes, scanning them, and often filling in missing data. But image-based solutions capture additional information about a package as well. “I have been working on 3D technology for dimensioning, which allows items of various sizes on a conveyor to be tracked,” Carey said. “In logistics, you want volumetric optimization, especially when you ship across a border, because at that point, everybody collects money. And it is possible to underpay or overpay, depending on what the calculations were before things were sorted and packaged.” If something has been packaged incorrectly when it is put on a truck, for example, the process of reshipping and repackaging requires time and money that companies don’t want to waste, Carey said. Lighting the way The importance of properly illuminating a high-speed imaging system cannot be underestimated, according to Steve Kinney, the director of engineering at Smart Vision Lights, which makes LED lighting for machine vision applications. LED systems can provide continuous or pulsating light within a narrower spectral range versus traditional xenon light sources. The company’s NanoDrive technology — featuring a driver and controller located in close proximity — can provide high-intensity light for extremely brief periods. A NanoDrive-enabled light configured to a 1-μs pulse width and a 10% duty cycle could deliver 100,000 strobes/s. The proximity of the driver and controller also minimizes latency. In practical terms, this means that the measure of LED output in intensity over time should form a square-shaped pulse when displayed in a diagram. This is critical to optimizing camera exposure times and avoiding image blur at high speeds. Kinney said industries that use Smart Vision Light’s NanoDrive come from many segments of the economy, including the pharmaceutical and packaging industries. “It’s not just a question of counting pills; the resolution needs to be there to look for breakages,” he said. “These days, companies are looking to track a product from the raw materials to when it appears on the shelf.” Kinney noted that on the manufacturing floor, customers want to be able to freeze a product in frame and not have lights flashing constantly. And, in some cases, sunlight can be a problem, so numerous companies are moving to red light or near-infrared light in the 850-nm range. “What happens is the sunlight can interfere with the lighting system, and people might start seeing rejections in the system,” Kinney said. “With light in this range for the cameras, it doesn’t cause as many problems. “There is a lot of knock-off lighting out there, but a system needs to be built with uniformity for sorting and controlling,” he said. “Many people end up thinking all you need to do is pick up your camera phone and read with a light, but we have all sorts of geometries of lighting systems because as the light comes to the lens, the way it diffuses is just as important as the field of view.” When various lighting geometries shine on the sorting line, sometimes the frame grabber will splice together three different images, which will then be combined into one image, Kinney said. This helps identify the tiniest of defects. To capture as much detail as possible, the camera will be positioned very close to the subject area and operate with a wide aperture. LEDs help to provide maximum lighting on the reduced depth of field. The resulting systems may also often use more of the spectrum, including UV LED and shortwave IR, which are more sensitive, Kinney said. “Some think they don’t need lighting to be effective because AI or deep learning will address that,” he said. “But, no, the lighting still needs to be good for everything down the line to work, too.” Imaging down the line One of the challenges of defining specific long-term goals in high-speed imaging is the different parameters assigned by different industries and markets. Two years ago, capturing 500 MB/s was an accomplishment for imaging. And now, 1 GB/s is what customers are looking for at a minimum, said Max Scholz, strategic marketing manager at Mikrotron, which provides high-speed imaging for troubleshooting and motion analysis in the automotive, packaging, food and beverage, electronics, and sports industries. Scholz said that one of the counterintuitive advantages of high-speed imaging is that it provides adequate data for slow-motion video recording, such as that used in automotive crash tests, or to break down sports replays or medical processes. “A camera can record from 1 to 6 Gbit/s, which can be used to analyze the images in real time without a frame grabber,” he said. “In many cases, companies will use this capability when they begin a project, to analyze the process and see what is happening, picture by picture.” High-speed barcode imaging proves invaluable in distribution centers where a variety of package types are rapidly processed for various destinations. Courtesy of Cognex Corp. Scholz further explained that in high-speed applications, megapixel sensors may capture a region of interest within the frame. At 10,000 to 30,000 bit/s, the imaging systems are able to follow laser applications such as laser triangulation, additive manufacturing, or laser welding. He said that applications such as the inspection of semiconductor wafers, printed circuit boards, and 3D-printing processes continue to present expanding markets for high-speed imaging. Camera resolutions running in the 20- to 25-MP range are important in many of these applications. Such resolutions put demands on cabling as well as the PC-based processing units that make up the bulk of the machine vision system. “We are working on a system that starts with a high-speed smart camera which can process data and deliver it to an open platform,” Scholz said. “From a customer’s perspective, they are looking for the cameras to be fast, and for the integration of the system to make everything work.”