Cameras are ubiquitous on the factory floor, and for good reason: Robots, heavy machinery, raw goods, containers and finished products fly around at breakneck speed, while fewer people than ever are slowing down production by watching over them. Pharmaceutical packages and the pills inside them. Glass bottles and the wine poured into them. Cell phones and autos with identification marks scribed into them. All of these and more have found themselves, at more than one stage of production, under the gaze of a rapidly blinking camera or other type of sensor. All of these cameras and sensors, in turn, are actuated by programs that tell them where to look, how long to gaze and what to do with the information they acquire. At a plant operated by KWD Automotive Group AG & Co. KG of Wolfsburg, Germany, robots do the heavy lifting (and welding). VisionPro 3D software helps by recognizing part position and possible distortions of the side panels in the supply rack and forwarding the data to a controller so that the robot can adapt its gripper movement to the actual position of the parts in real time. Two robots work in conjunction to gauge the initial position of the parts, grab individual parts, and place them in the correct position and in the logical sequence. After welding is completed, the robots perform an optical spot weld inspection, gauging the number and location accuracy of the welds. The 3-D vision system checks whether the welds were placed in the predefined fields. Courtesy of Cognex. Even in a generally down economy, manufacturing is on the uptick in many regions, including the US. Some of the increase can be related to a bump up in the adoption of machine vision systems and the software that runs them. Machine vision on the whole grew by nearly 35 percent in 2010, according to the European Machine Vision Association, which expects to see growth of another 20 percent this year. One size fits most Machine vision imaging systems offer a wide variety of tools and programming tricks to entice users in a wide range of industries. “We have customers using our EasyGauge subpixel measurement library to measure integrated chip leads in Taiwan, to gauge fish in Northern Europe and to inspect pharmaceutical vials in the Middle East,” said Marc Damhaut, CEO of Euresys in Angleur, Belgium. “Of course, the way the library is used – the way it is integrated into the whole vision system – depends on the application.” A fruit sorting and inspection system uses the Matrox Imaging Library to process images of apples. Binarization and blob analysis separate the fruit from its background and obtain dimensions like Feret diameters and elongation. Courtesy of Matrox Imaging. Cameras on the production line often are used to inspect products at various stages of the manufacturing or assembly process. For example, a flat roll of photovoltaic thin film is constantly scanned for cracks, separations and other defects as it lumbers on from roll to roll on the line. Molded plastic bottles can be checked for defects as well – as they are produced or as they are filled with products ranging from aspirin to fruit punch. A good algorithm will help find a misfilled bottle, a loose cap, or a skewed or misprinted label. If the software driving the inspection system is clever enough, it can classify the types of defects it finds, not merely catalog them, and direct a robot arm to kick the bad product out of the line. In the production process used by pharmaceutical giant Boehringer Ingelheim GmbH of Ingelheim am Rhein, Germany, blister packs pass over a narrow belt at a speed making it impossible for the human eye to see any more than the outline of the individual objects. Instead, state-of-the-art vision technology ensures error-free quality. Depending upon the product type, examination is made of varying details, such as lot number and expiration date. Cognex’s VisionPro-based software recognizes all the relevant symbols and letters and, on the basis of previously taught parameters and tolerance limits, assesses the quality of a product. Courtesy of Cognex. But just as there is a vast array of parts manufactured and final products assembled, there is an equally large number of imaging tasks that must be handled, sometimes by the same camera-and-software combo. Euresys, National Instruments, MVTec Software, Cognex and others offer top-end software systems featuring large suites of tools designed to make identifying and programming imaging tasks easier for the end user. Such imaging suites tend to include modules for groups of tasks that include the following: • preprocessing images; • matching patterns within a single image or between multiple ones; • measuring between points on a part; • identifying points or regions of interest; • locating parts or defining their shape; • analyzing color, text and position of parts, labels or markings; • separating images into discrete objects, or blobs. “People new to machine vision technology might have a small amount of difficulty understanding the software that drives it, but it is not a big problem,” said John Petry, a senior manager at Cognex Corp. of Natick, Mass. Many of these tasks – and much of what machine vision software strives to accomplish in general terms of functionality – have been set in stone for some time now. The major current challenges are keeping up with customer demands and the ever-evolving computers that are running the show. One of the key processes at Austria-based auto battery maker Banner GmbH’s finishing plant is the visual final inspection. Based on Cognex’s VisionPro software, the image processing software OBI (Optical Battery Inspection) from Schmachtl GmbH coordinates the data from six industrial cameras and one Cognex DataMan 100 ID reader. The OBI system inspects the batteries from five sides and verifies the position and completeness of the labels, various battery dimensions, the presence of pole caps and grips, and the dimensions of the vents. The software also reads the label pinned directly onto the plastic housing of the battery, which contains information about the battery type, manufacturing date and performance. Courtesy of Cognex. When most of the available software began to undergo development, chiefly in the 1990s, PC processors were relatively rudimentary. Now, though, multithreaded, multicore chip sets have become the norm as Intel, AMD and other chip makers have continued to pack additional processing power into shrinking volumes. Image capture and input/output functions, for example, can be run simultaneously on separate processor cores – increasing the overall processing speed, which in turn enables production lines themselves to roll on faster rates. To achieve this state, however, the software has had to undergo several stages of evolution so that the growing numbers of cores (or threads) are not wasted. Automated optical inspection systems – as used in different manufacturing industries – do share common underlying requirements, such as locating objects or features, analyzing/measuring objects/features, and reading/verifying ID marks, said Pierantonio Boriero, product line manager at Matrox Imaging in Dorval, Quebec, Canada. Other systems have other requirements, however. Medical imaging systems, for example, differ substantially in that their goal is to enhance visualization to aid diagnoses. Typical users of imaging software favor tools that can be simply configured and controlled without an in-depth knowledge of the mathematics and signal processing techniques employed by the software and hardware. Users like having a large diverse set of software tools because it enables them to tackle new applications with confidence, Boriero said. Users like complementary tools, he added, which enable them to look at alternative approaches to the same application, and they want tools that get the most out of state-of-the-art hardware, such as multicore CPUs. From fish to pharmaceuticals, Open eVision’s subpixel function gets the right measurement. Courtesy of Euresys. Open eVision Euresys originated Open eVision in 1990 as a set of embedded image-processing functions for the company’s first acquisition cards based on digital signal processors. The company transformed the package into a PC-based library in 1996 and began adding functionality for image analysis, such as subpixel measurement, pattern matching, blob analysis and optical character recognition, or OCR (at first). Since 2000, the company has added color analysis, 2-D code reading and optimization for use with the latest CPU architectures, among other features. “Surprisingly, the most popular features of Open eVision are still the first libraries that we have developed: subpixel measurement, blob analysis and pattern matching,” Damhaut said. “They are still appropriate for most of today’s applications.” Matrox Imaging Library Introduced by Matrox Electronic Systems Ltd. in 1993, the Matrox Imaging Library and its core application programming interface have remained essentially the same ever since. The package has had to adapt over the years as operating systems, programming languages and video interface technologies have evolved, Boriero said. Gray-scale values help determine the quality of cookies. Geometric pattern recognition locates the cookies for placement inside the trays. Courtesy of Matrox Imaging. “Requirements from users and their applications have prompted the development of higher-level tools that are simpler to configure and deploy, flexible and robust,” he said. Geometric pattern recognition and feature-based OCR are recent examples of the software’s increased capabilities. “Both the evolving needs of our established user base and the requirements of emerging applications/industries for digital image-processing technology steer our product development plans,” Boriero added. VisionPro VisionPro is about 10 years old, having started as a C++ platform to let OEMs jump-start the imaging products they were using; the software library at the time worked with only a select number of cameras and frame grabbers. But as the company began working more directly with end users (manufacturers with factories to optimize), the software had to grow, Petry said. Better programmability and better user interface were required, he added. Now, VisionPro works with just about every possible camera and frame grabber as well as with many other sensors, such as thermal imagers. The company’s biggest targets for VisionPro are systems integrators – companies charged with installing large, complicated assortments of cameras, sensors, robots and more. The software is well-suited for systems integrators who, according to Petry, particularly like the package’s capacity to multitask: to read characters and perform quality control checks simultaneously, for example. The VisionPro 3D startup kit from Cognex. Courtesy of Cognex. Petry said that advances in VisionPro have come in step with changes in Microsoft Windows; for example, multithreading capability is now standard. 3-D and the future of imaging Of all the vision tools now available, 3-D imaging is seeing the greatest increase in demand. Robot-based manufacturing is driving a lot of the interest in 3-D because it enables more complex motions in any given volume of space in which a robot has to maneuver. Robot-based parts picking, welding, marking and most other tasks aren’t too difficult to program, but getting a robot to put two or more parts together is very hard. Unfortunately, according to Petry at Cognex, easy assembly (primarily via the software end of things) was overpromised. Several integrators went out of business trying to get the multicamera systems required for 3-D-based robotic assembly lines figured out. Now, though, more powerful PCs can crunch the numbers needed to make 3-D systems viable. More changes are coming in the 3-D imaging segment of the vision industry, but they will come in small increments. As for the prospects for machine vision software, there are multiple paths to success. For example, there is huge growth in China, according to Petry. Once thought of only as a source of low-end manual labor, the country has exploded in the high-end manufacturing capacity, and companies there are now using sophisticated imaging systems on a regular basis. Euresys’ Damhaut cites LED and solar panel inspection as growth areas for imaging analysis. “We also see a diversification of the applications toward nonmanufacturing industries,” he said. Cognex is “constantly releasing new vision tools,” Petry said, citing new optical character reading tools as a recent example. The challenge, he added, is to find the next feature to focus on.