From detecting flaws to verifying the accuracy of Braille printing on packaging, machine vision’s role in the COVID-19 era comes to light.
Steve Zhu, TELEDYNE DALSA
By nature of what is at
stake, pharmaceutical and medical industries have
developed much stricter standards compared to most other sectors. From day one, quality control and track-and-trace processes have been paramount for pharmaceuticals, and the COVID-19 pandemic has underscored the
importance of vaccine quality as a global health issue.
Makers of COVID-19 vaccines require strict quality inspection at every step. Courtesy of iStock.com/Avatar_023
In the pharmaceutical industry, the same typical applications of machine vision inspection are performed as in other industries, but strict standards and specifications are defined as needed for each particular task. So, as a function of quality control, manufacturers must ensure that every step of production — from product seals to capsule sizes to lot numbers printed on bottles — is within the specified standard range, especially from a safety and security point of view.
Quality control can be composed of several types of vision inspection. The basic categories include:
• Presence and absence detection.
• Flaw detection.
• Verification — 1D, 2D, and OCR
(optical character recognition).
• Measurement.
• Robotic guidance.
Presence and absence detection. Presence and absence detection checks for a certain attribute and whether it is seen or not seen by the vision inspection system. This type of inspection is often used in packaging for pharmaceuticals or a medical
device. The manufacturer needs to make sure that each product has been packaged properly and that every part or piece is in the right position and nothing is missing. Presence and absence detection can also be used during the manufacturing process to make sure that every piece of a product is properly installed. Such detection could include use of a pattern match to verify whether the trained pattern is present or absent. Another example involves the use of a count tool that can tally parts and record their locations.
Flaw detection. Surface flaw detection tools can identify scratches and cracks on surfaces such as glass and metal. This type of detection is extremely helpful in the inspection of medical devices, where early detection of flaws can save on production costs. Flaws can be recognized in the form of contamination, such as specs of dust or dirt in vials or medical bottles.
Verification: 1D, 2D, and OCR. In the U.S., the FDA’s Barcode Label Requirements require products to be labeled with a linear barcode that is used to
encode National Drug Code (NDC) identifiers. The Drug Supply Chain Security Act, enforced by the FDA, requires pharmaceuticals to be marked with a linear barcode carrying the product NDC. The act also requires products to be marked with a data matrix barcode that carries the NDC, serial number, lot number, and expiration date.
To ensure this information is clearly printed and readable, scanning tools that read 1D barcodes and 2D matrix codes (such as QR codes), or that perform OCR, are used to read the letters and numbers on a label.
Often part of an integrated vision solution, a machine fitted with these tools could inspect 1D and 2D codes to make sure the expiration date is correct, printed clearly, and positioned correctly. These scanning tools are used for traceability, sorting, and process control in the pharmaceutical industry. They are essential for verifying product date and lot codes and package unit dose codes, and for fulfilling the track-and-trace requirements throughout the pharmaceutical supply chain, along with compliance with the FDA’s 21 CFR Part 11 regulations.
Presence and absence detection is often used in assessing packaging for pharmaceuticals. Courtesy of iStock.com/Neznam
Measurement. Nearly every industry uses measurement as a machine vision quality inspection tool. In the pharmaceutical industry, accurate measurements are required for applications such as measuring the size of medical needles or a surgical knife. So, machine vision is needed to precisely measure the shape and the size of a part. If any part measurements are out of the specified range, the consequences could be life-threatening. Measurement tools can also be used to determine fill levels of a medicine bottle or vaccine syringe, for example, or to verify the presence of a safety seal.
Robotic guidance. Along with
inspecting objects, machine vision can position them via robotic guidance.
This process includes locating and
verifying parts for robot-guided pick-and-place applications and orienting parts on an assembly line. In the
pharmaceutical industry, a positioning tool could ensure the right cap alignment on a medicine bottle or position
a rubber seal correctly in a syringe.
These vision inspection functions typically do not stand alone. Often, two or more will be used together in an integrated inspection solution. Systems integrators, such as Shanghai Botrong Electric Co. in China or Bestell Solution in Singapore, create specialized vision solutions to solve specific inspection challenges that are facing the pharmaceutical industry. Machine vision
components — including cameras,
illumination, software, and PCs, along with conveyors or robotic systems — move the products or packages into place.
Packaging inspection
Like all vaccine manufacturers, Beijing-based biopharmaceutical company Sinovac Biotech faced pressure to produce safe, high-quality vaccines. As the maker of CoronaVac, which was developed to protect against COVID-19, the company is rolling out some 300 million doses in its newly built 20,000-sq-m production plant, according to the BBC.
To achieve these production goals, the company turned to machine vision inspection to meet these demands for the packaging and tracking of vaccines. Machine vision cameras, together with vision software, are part of a system that inspects the vaccine packaging, checks the label and tracking information, and verifies that the right vaccines are placed in the right boxes. Vision is also used to inspect the vials themselves.
With the advanced shape-from-shading algorithm and some preprocessors in Teledyne DALSA’s Sherlock software, the unwanted complex background characters have been so significantly suppressed that Braille can then be easily identified and reliably read. Courtesy of Teledyne DALSA.
The CoronaVac vaccine is inactivated — meaning it is not an mRNA-based vaccine such as those produced by Moderna and Pfizer-BioNTech. This means it can be stored in a standard refrigerator, rather than requiring extremely low storage temperatures. (Moderna’s vaccine must be stored
at −20 °C and Pfizer-BioNTech’s at −70 °C.) The ability to use standard refrigeration gives the Sinovac vaccine
advantages in terms of storage and transportation costs and makes it much more realistic for use in developing countries.
The Sinovac vaccine packaging is inspected using Teledyne DALSA’s Sherlock imaging software interface connected to four Genie Nano cameras (two 5M GigE Vision cameras and two 1.3M GigE Vision cameras) and a GEVA vision system to support multicamera
inspections. The lot numbers and barcodes are read to confirm that the vaccine information is correct and legible on the packaging. This information reveals the vaccine type, its manufacturer, content, packaged quantity,
lot number, and production date, and it can be used to trace shipments through the supply chain.
Braille printing
Poorly printed medical instructions on drug containers could expose
patients to great risks, such as missing a refill date, swallowing the wrong pill, or ingesting expired medications. These risks can be especially challenging to people who are blind and have
to read Braille to understand the
prescription name, dosage instructions, and other label information.
Braille dots, by nature, are difficult
to read. At only 0.2 mm high and
embossed across already printed
characters, it is extremely challenging for most vision technologies to inspect
the quality of Braille. Furthermore,
deciphering the Braille against the background adds to the complexity.
Teledyne DALSA developed a special Braille vision inspection system by applying shape-from-shading technology using the company’s VICORE smart vision system with area-scan cameras, together with the Sherlock shape-from-shading algorithm to get a high-contrast 3D-effect image from the Braille out of a complex background. Sherlock then applies preprocessors to optimize the imperfect shape of Braille dots to allow the OCR algorithm to read the Braille characters.
This inspection system has ensured the quality of printed Braille on medical drug containers so that the dose instructions can be consistently and reliably read correctly by people who are blind or visually impaired.
Artificial intelligence
Artificial intelligence is beginning to support many facets of machine vision inspection. Using processing tools to perform AI inference on models, tasks such as classification, anomaly detection, object detection, segmentation, and noise reduction can all be optimized based on machine learning from vast amounts of image data.
One application of AI is to support OCR reading. For instance, due to poor printing quality, it is very challenging for most conventional OCR algorithms to read every character reliably. When OCR is integrated with AI technology, this inspection task can be handled perfectly, without having to train the system on all the characters.
The pharmaceutical market is one of the most demanding industries in terms of identifying, sorting, and verifying products — requiring the strictest
quality control. Quality inspection mistakes in manufacturing, packaging,
or end-to-end traceability can lead to injury or death. This emphasizes the criticality of using the best inspection
tools and methodology available,
including newer technologies such as 3D inspection and AI.
Meet the author
Steve Zhu is Teledyne DALSA’s director of sales for Asia. Utilizing his 25 years of experience in the machine vision, automation, and fiber optic communications industries, Zhu works closely with customers to solve their complex machine vision challenges; email:
[email protected].