Medical device manufacturing encompasses products that go inside the body, such as catheters and artificial joints. Other manufactured goods in this category include external devices that measure vital signs and other medically important parameters, such as pulse oximeters that capture blood oxygen levels or EKG machines that detect heart activity. In all cases, doctors and patients count on manufacturers to deliver products that work, whether used inside or outside the body. This has implications for vision systems used in quality control, process monitoring, and other inspections. For instance, a vision system designed to inspect an artificial joint or another medical device for minute imperfections would require the ability to spot flaws that are quite small. Medical devices, however, typically aren’t made in high volumes, compared to everyday consumer goods, said Tom Brennan, president of Artemis Vision. The Denver-based systems integrator builds machine vision systems for industrial control applications. According to Brennan, the specifications that medical devices must satisfy are often tight, requiring precise dimensions, specific printed documentation, part-number tracing, and high quality. Meeting such specifications may turn an inexpensive 0.1-cent plastic part into a 10-cent one, with this hundredfold increase arising in part from such factors as decreased yield due to parts being rejected for not meeting requirements. “You’re inspecting quality into the part, and you’re increasing the price because it’s to a guaranteed quality,” Brennan said. Machine vision helps medical device manufacturers get products out the door as part of inspection systems. Medical devices are often manufactured for years using a process that cannot change without regulatory approval. So inspection systems perform quality checks that ensure that the devices meet regulatory requirements — a task that can necessitate extensive engineering and a novel approach. Finding defects Artemis Vision provides machine vision solutions to medical device manufacturers for products that are used external to the body, Brennan said. These vision systems can perform precise inspection that spots and helps weed out functional defects that have possible serious medical and legal implications. Classifications of parts into categories of good or bad can be challenging because the difference that counts as a defect is often subtle. Fixturing ensures consistent and proper part alignment for inspection during medical device manufacturing. Reliable part placement helps ensure optimal machine vision results. Courtesy of Artemis Vision. A component used in knee or hip replacement surgery, for instance, may have pits or bumps that are so tiny that they are not likely to affect the operation of the device. Or bags used in medical devices might have a hair or other small foreign matter in them. Reliably catching small pits or foreign matter often requires a resolution that is significantly finer than the deficiency itself. Spotting and categorizing a 100-µm bump or pit, roughly the diameter of a human hair, could entail a vision system resolution of 25 µm, or even 10 µm, per pixel. Inspecting to this level of detail can be expensive and time-consuming. Although these faults may be effectively cosmetic, small blemishes could nonetheless be important to detect because of how the end user might view a less-than-perfect component, particularly one going into the body. “If I’m the patient or the surgeon, then I don’t want that one,” Brennan said. Because there is often a need for this type of fine inspection capability, vision system resolution and the resulting ability to spot defects is important. Thus, highlighting the defect with lighting of sufficient contrast and achieving proper positioning of the part play key roles in ensuring vision inspection success, Brennan said. Often, the vision inspection tasks are similar to those found in other types of manufacturing. Some parts must be checked for quality, a job initially performed by people, who typically cannot maintain attention and focus for long. Indeed, inspecting a part manually at high magnification can feel like looking out of the window of a plane flying at low altitude over changing terrain, potentially causing motion sickness. The manual inspection situation is made more problematic when people are called upon to check their own work at a hand-assembly station. This situation leads to the potential for out-of-spec parts to slip through. Brennan said he has seen this scenario play out several times. Machine vision inspection provides a solution that is repeatable and eliminates human error caused by fatigue, and thus is better able to spot foreign particulates in a fluid-filled container, for example. A 3D-printed prosthetic jaw (1); a close-up shot of a prosthetist assembling a prosthetic leg (2); various explanted pacemakers, defibrillators, and event recorders (3); a rendering of a 3D printer printing a prosthetic spine (4). Machine vision-based inspection can also check other parts of the manufacturing process, such as verifying that welds are done on both sides of a part. This can be a critical step to ensuring that medical devices will meet performance requirements. While many aspects of such inspection tasks are common to all manufacturing, some are unique to medical device manufacturing, said Felix Chemnitz, medical product market manager for Ahrensburg, Germany-based computer vision imaging component maker Basler AG. Consider the example of a medical device inspection that needs to be performed over a long period of time. The device may be manufactured using virtually the same process for decades so that its maker avoids repeating the regulatory approval process unless absolutely necessary. The implications for cameras and sensors is that they may be replaced repeatedly over that span. “Long-term availability is always important,” Chemnitz said. He said vision systems perform a number of tasks in medical device manufacturing, such as monitoring production, checking the quality of parts as they are processed, and verifying that parts have the correct unique identification on them. Faster imaging speed results in higher throughput, which benefits all of these tasks. One way to increase the imaging speed is by using a faster interface. For this reason Basler is deploying CoaXpress 2.0 in its newest cameras. The new standard offers up to 12.5-Gb/s data rates per cable — double the transfer rate possible with CoaXpress 1.11. The ability to move data faster through the interface could substantially cut inspection times. Increasing camera resolution also pays off in various ways. If finer imaging is needed, then higher resolution can help. According to Chemnitz, standard resolution across the Basler product line was 5 MP five years ago, and today resolution is as much as 25 MP. All things being equal, having more pixels improves resolution. Increasingly dense printed circuit boards found in medical devices have smaller features and so require higher-resolution imaging for inspection tasks. Courtesy of Basler. Greater resolution can provide an advantage in other situations that do not call for imaging smaller features, according to Chemnitz. Suppose that an inspection uses several cameras. It may be possible to replace them all with a single higher-resolution system, potentially boosting throughput and cutting cost. Verifying part identification Chemnitz said medical devices are, in general, getting smaller and smaller when it comes to verifying part identification. Hand-held devices such as portable medical-grade EKG units or digital hand-held fundus cameras for retina diagnostics, for instance, are a growing part of the market. These devices are full of electronic parts, including jam-packed printed circuit boards. Cameras must inspect these small electronics, a task that could demand high resolution and magnification. This need is also found in consumer goods, so the same vision solution could be applied. A difference with medical devices, however, is a possible requirement to track what goes into a device at the component level. This is where the unique identifier comes in, with one example being a 2D barcode. This matrix of lines and spaces has to fit on the part, and in the future, as components get smaller, imaging this code will become more challenging. “It will be a really, really small code, and you still need to be able to read this using the camera,” said Chemnitz, who already sees solutions in place to handle this task. The highly regulated nature of medical devices can affect inspection during their manufacturing, said Robert Eastlund, president of Graftek Imaging. The Austin, Texas-based company is a systems integrator that sells solutions to customers in a wide array of industries. Due to rules that mandate that no changes be made in the operation of a medical device, manufacturers may require that there be no changes to inspection protocols. In the past, this could have been handled by going with a unique camera part number specific to a particular application. Now, Eastlund said, some camera vendors, including Basler, have split off their medical application cameras as a separate and relatively static product line. Inspection tasks performed during medical device manufacturing might involve determining dimensions in real-world units such as millimeters. This is achieved by correlating the size of a pixel on the sensor to the measurement it represents on the imaged object. It then becomes important to calibrate camera pixels to the chosen measurement units. Eastlund said that using a telecentric lens is one way to ensure repeatability when determining dimensions during an inspection. Finally, he said there will always be medical device inspections that are impractical and not affordable. Carrying out a quality-control check of an irregular or amorphous object, for example, cannot feasibly be done by today’s machine vision. However, customers keep making such requests, Eastlund said. Advancing technology, particularly in machine learning, could make these inspections possible in the future. Without such advancements, though, Eastlund said of satisfying some inspection requests, “It may not be practical at this time.” Embedded Cameras While usually performed during medical device manufacturing, some machine vision inspection takes place during medical device operation. A case in point comes from Cognex. The Natick, Mass.-based supplier of vision systems has a line of medical embedded vision products that are similar to traditional solutions but in a more compact form that is attractive to machine builders and OEMs. In a fundamental way, these embedded solutions are like everything else the company does. “Cognex’s bread and butter is vision inspection,” said Jörg Vandenhirtz, senior business development manager for life sciences for the company’s ViDi Deep Learning and Advantage Vision Engines (Europe) division. To provide an example of vision inspection during operation, he pointed to the PerkinElmer Janus G3 Blood iQ system. This medical device automates genetic analysis of blood. In many respects, it functions like a compact factory, handling perhaps 10,000 small blood-filled tubes daily. Machine vision and deep learning enable multipoint aspiration of blood samples for genetic analysis. Courtesy of Cognex and PerkinElmer. Embedded cameras inspect and analyze these tubes, running presence/absence checks to ensure the tubes are where they are supposed to be. Cameras also look for fluid levels, even going so far as to differentiate among and pinpoint the location of particular types of blood cells after a sample has been centrifuged. Armed with such information, the system can extract white blood cells, for instance, and then analyze them. Another vision task is to simply determine tube size. No standard exists, and the system needs to know the dimensions of the object it is dealing with. Vision systems can provide this information. From an inspection perspective, this application presents challenges. The cameras must be small because they have to fit within the tight confines of the available space while staying below allowed heat dissipation levels. At the same time, the camera needs about a 10-cm field of view — the general size of sample tube. Inspecting an entire rack of tubes requires about a 50-cm field of view. Another challenge has to do with the object that is being analyzed: blood. Centrifuging, for instance, separates blood into red and white cells as well as plasma. However, the exact location of each layer, which can be quite thin for white cells, varies from tube to tube. “You have the typical biological variation, as well as tube and package variation, so embedded deep learning is the way to go,” Vandenhirtz said. Machine vision solutions help automate the operation of genetic blood analysis. Courtesy of Cognex and PerkinElmer. In deep learning, software is shown good and bad images, using these to derive categorization criteria. However, in medical applications, the overall machine may have to be cleared by regulators for use, which may mean no significant aspect of the system can change. So, freezing the learning process and firmware before putting the machine through regulatory clearance may be necessary. Whether this is the case or not will depend on the particular situation. Vandenhirtz said that deep learning and vision inspection in embedded cameras will become increasingly important and common-place in medical devices that operate like miniature factories.