AI Raises the Bar for Package Inspection

In combination with package inspection or OCR, deep learning can enable vision systems to effectively handle variations in the imaged object.

SZYMON CHAWARSKI, TELEDYNE DALSA

Package inspection has come a long way, due to traditional machine vision methods that use image capture, processing, and analysis to automate quality control. These methods are crucial for ensuring process quality, improving production efficiency, and maintaining compliance with industry standards. While traditional methods have been effective, advancements in artificial intelligence (AI) and machine learning are enhancing the capabilities and accuracy of machine vision systems in packaging inspection, aiding the arduous task of analyzing visual data.

Courtesy of iStock.com/SDI Productions.

Courtesy of iStock.com/SDI Productions.

Through the full life cycle of any package, a wide variety of machine vision software and hardware can be used to ensure favorable quality. Machine vision is applied during the manufacturing of the package itself and used to guide automation when filling it with products and verifying the contents. After the package has been filled and sealed, machine vision is also used to read barcodes and labels to track and trace the box as it travels through the supply chain.

Boxes and bottles

Consider the ubiquitous cardboard box, the most common item that defines the packaging sector. In the U.S. alone, more than 80 billion corrugated boxes are used each year, and ~80% of products sold are packaged in cardboard, according to All County Recycling. A cardboard box is made from a flat sheet of cardboard and folded into shape. During the manufacture of the box, cameras take pictures of all sides and machine vision tools locate the edges and corners. The vision system quickly measures the size and parallelism, and verifies that the box has been manufactured to specification. These checks are typically performed with multiple cameras pointing at each side of the box. Task-specific machine vision lighting is usually integrated into the system to ensure that the images are repeatable. Boxes pass the cameras at a high rate of speed and get measured quickly and accurately.

The box is a simple example, but the principle can be applied to more complex shapes, for example, blow-molded bottles. Machine vision systems check plastic bottles for weak spots where the plastic appears discolored, cracked, or thin. They also check for excessive flash material or dents — the sort of defects that indicate upstream machinery problems. Such checks can help alert operators of impending failures.

A Teledyne DALSA Genie Nano camera is used on an automated picking and packing line to track and identify packages. Courtesy of Teledyne DALSA.

A Teledyne DALSA Genie Nano camera is used on an automated picking and packing line to track and identify packages. Courtesy of Teledyne DALSA.

A wide variety of machine vision algorithms catch defects in blow-molded containers. If a system is inspecting a simple bottle shape, and potential defects are easily visible, a vision system uses a traditional stack of image processors, such as adaptive intensity thresholds, combined with blob detectors. For more complex inspection problems, AI machine vision classification or segmentation models can be deployed.

Filling the packages

Vision systems are also widely used further downstream in the packaging process. Boxes, bags, bottles, and other containers must be filled. Machine vision is used to count the number of items placed in a box. The algorithms required to do this do not need to be complicated or cutting-edge; for example, if the system needs to count how many bottles have been placed in a box, it can use a blob tool that identifies boundaries of areas with similar colors. The blob tool can count the bottle cap areas and easily return the number of items present.

For more challenging applications in which objects have complex and overlapping shapes, AI algorithms can be used to accurately count and identify the objects that are present. Using an object detection model, a vision system can identify prescription pill bottles placed in a box. The model is trained using annotated image data from example pill bottles, and when the model runs in the field, it can count bottles accurately in any orientation even if they are partially hidden or overlapping, or inside different box sizes and orientations.

OCR to read letters and numbers

Optical character recognition (OCR) uses pattern-matching to recognize and read printed letters and numbers on packages. OCR algorithms can be taught by the end user by using examples of text, or they can come pretrained on a wide variety of texts, colors, and print surfaces. In addition to reading characters, some software packages also contain optical character verification (OCV), which is used to verify the print quality and legibility of text. OCR and OCV are particularly useful in packaging to read and verify expiration dates, lot codes, and serial numbers. When OCR/OCV are combined with other machine vision tools, they can be an important and powerful solution for traceability and quality control.

A machine vision algorithm identifying a cracking failure on a blow-molded bottle production line. Courtesy of Teledyne DALSA.

A machine vision algorithm identifying a cracking failure on a blow-molded bottle production line. Courtesy of Teledyne DALSA.

A simple and fast machine vision blob tool used to count bottles inside a box on a production line. Courtesy of Teledyne DALSA.

One benefit to using OCR compared with other inspection methods is that it aligns with the way customers interpret information. Data could be printed using barcodes or other types of labels, but by using and verifying text, labels and packages are checked in a human-readable way. The use cases for OCR/OCV are very broad and in some cases extremely important. For example, food packaging is required to contain an accurate expiration date. Medicine and medical devices must include lot codes and critical specs and dosages on their packaging, which must be accurate to maintain safety and compliance.

Like all machine vision algorithms, OCR is benefitting from the growth of deep learning and AI. OCR algorithms can leverage AI detection models to read characters in difficult situations. When applied to package inspection or OCR, deep learning can enable vision systems to better handle variations in the imaged object. For example, applying a deep learning algorithm for OCR improves readability even if the label is damaged or partly hidden. This can be useful for countering environmental interference, variable printing quality, and other uncontrollable variations in the application. When customers read a label, they can usually understand misprinted text using the context and filling in gaps. AI OCR models have a better ability to mimic human perception than traditional methods that require rigid pattern structures.

Edmund Optics - Manufacturing Services 8/24 MR

Barcode scanning

Barcode scanning is another key technology in package inspection. Barcodes offer standard methods to encapsulate text and number data inside a small pattern. Machine vision tools can be used to quickly locate and read barcodes in an image. One-dimensional (1D) codes, such as universal product codes (UPCs), and two-dimensional (2D) codes, such as QR codes, are widely used in the packaging industry. These codes can contain serial numbers, lot codes, stock-keeping units (SKUs), and other product information and present them in a standard format that can be read throughout the packing, shipping, and sales product cycle.

Barcode scans can be performed rapidly to keep up with fast production lines. Like OCV, most barcode machine vision tools have print-quality grading metrics that can be used during the inspection process to verify that an expected code has been printed correctly, checking that upstream labeling and printing equipment is working accurately. To read barcodes on packages, multiple cameras are often placed on a production line, looking at packages from all possible angles. The cameras are synchronized and can quickly find and read a code in any orientation. Once the barcode is read, a vision system can classify and sort the package, possibly ejecting it from the production line if it is incorrect or damaged. Most vision systems can be integrated with enterprise resource planning software and can add valuable tracking and traceability functions to a production line.

3D sensors for dimensioning

For packaging applications, advanced 3D vision sensors are another technology used in package inspection. They offer innovative solutions for high-speed dimensioning and location of packages. The traditional approach was to use multiple cameras placed around target areas and measure a box or pallet length, width, height, location, and angle by combining data from all the cameras. Currently, it is possible to use a single overhead 3D sensor and capture all the required data at once. A 3D sensor can capture the dimensions and location of a package or item and pass that information to a robot arm that can process it in a high-speed automated manner. The high-speed aspect of this use case is crucial in environments in which large volumes of packages must be processed rapidly. These types of automated packing systems can greatly reduce the human manual labor cost of moving and handling packages.

AI aids traditional approaches

AI technologies are transforming packaging inspection by addressing issues that traditional methods alone could not resolve. AI offers a path for faster and easier machine vision deployment by allowing anyone with minimal experience to train inspection models. These AI-driven systems handle the heavy lifting by processing complex visual data and identifying subtle defects that may be missed by human inspectors or conventional vision systems. In packaging applications, AI excels at identifying and tracking packages or finding defects in packages with random orientations, shapes, colors, and surface finishes. This capability is critical to ensure high standards of production line reliability, quality, and compliance. As AI solutions mature and become easier to use, they will help drive more efficient and cost-effective packaging processes.

An AI object detection algorithm identifying bottles of different sizes and shapes inside a package before shipment. Courtesy of Teledyne DALSA.

An AI object detection algorithm identifying bottles of different sizes and shapes inside a package before shipment. Courtesy of Teledyne DALSA.

The inspection of aluminum cans on a production line using a Teledyne DALSA Genie Nano camera and ring light, aided by Astrocyte AI software. For details, see the December 2023 issue of Vision Spectra. Courtesy of Teledyne DALSA.

In the realm of sustainability, AI plays a significant role in improving the recyclability of packaging materials. For example, Eagle Vizion’s machine vision system inspects shredded plastic particles to ensure that they meet quality standards for recycling. This AI-driven approach helps in sorting and processing plastic materials more efficiently, ultimately reducing the environmental effect of packaging waste. The vision system’s ability to accurately identify and classify different types of plastic particles ensures that recycled materials are of high quality and suitable for reuse. This not only supports environmental sustainability but also aligns with regulatory requirements and consumer expectations for eco-friendly packaging solutions.

Improving accuracy

In summary, traditional machine vision methods have long been integral to packaging inspection, ensuring product quality, efficiency, and regulatory compliance through a combination of image acquisition, processing, and analysis techniques. Today, the advancement of AI and machine learning has significantly enhanced these capabilities, offering improved accuracy and efficiency. AI-driven systems complement traditional approaches by handling complex data analysis and identifying subtle defects that might be missed by conventional methods. The integration of AI technologies streamlines operations, reduces waste, and maintains high standards of product quality.

For plastic bottle recycling, particles undergo a multistep recycling and inspection process, aided by Teledyne DALSA’s Genie Nano cameras and Astrocyte AI software before they are reused in new plastic products. Courtesy of Teledyne DALSA.

For plastic bottle recycling, particles undergo a multistep recycling and inspection process, aided by Teledyne DALSA’s Genie Nano cameras and Astrocyte AI software before they are reused in new plastic products. Courtesy of Teledyne DALSA.

Real-world applications, including the inspection of boxes, aluminum cans, and the recycling of plastic beverage bottles, demonstrate AI’s potential to address complex challenges. AI models can now be trained with significantly less data, making them more accessible and cost-effective. As the packaging industry continues to evolve, the synergy between traditional machine vision and advanced AI technologies will play a crucial role in meeting the demands for quality, efficiency, and sustainability.

Meet the author

Szymon Chawarski is a product line manager at Teledyne DALSA with 15 years of experience in computer vision, automation, and product design. He leads new product development for industrial vision systems and smart cameras, blending hardware design and software development to create user friendly and cost-effective machine vision products. He has recently focused on applying AI and deep learning to solve challenging real-time inspection applications; email: [email protected].

There are 240 suppliers of Vision Systems in the Photonics Marketplace.

About Teledyne DALSA, Machine Vision OEM Components