Hank Hogan, Contributing Editor
For smart cameras, image isn’t everything. There’s also image processing to consider. Being able to quickly categorize a pill as the right shape or color, for instance, may make the right type of smart camera suitable for pharmaceutical work. To meet the demands of specific applications, vendors are boosting the image processing intelligence of smart cameras.
For example, Japan’s Omron Corp. soon plans to announce new pattern-recognition capability for such devices. Improvements like this not only make smart cameras better at finding the right part for a robot to pick up and more accurate when ensuring that a bottle is filled, but also enhance their ability to make certain that the right label is correctly attached to a product.
The three big trends in smart camera image processing include different processor schemes, improved connectivity and better hardware. The last two provide more power and flexibility, while the first looks at things in a new way. An example is the neural network approach taken with the ZiCAM line by Sunnyvale, Calif.-based Pulnix America, which was recently acquired by JAI of Copenhagen, Denmark.
“Neural networks are a little bit different than other kinds of processors, in that they’re not particularly good at metrology,” said Mike Kelley, the company’s smart-camera business director.
Introduced late last year, the ZiCAM uses a 312-neuron network. During training, the weight of the neurons is adjusted. When presented with captured data, the neurons vote and produce an image processing result. No programming is involved, although there is an adjustable threshold parameter.
According to Kelley, the ZiCAM is good at tasks such as determining if someone is standing in a doorway, detecting the level of liquid in a bottle, or finding an alignment mark or feature on a part during assembly. It’s not good, however, at precise measurements, such as finding the diameter of a bore shaft.
JAI has taken a different approach. The company’s flagship smart camera product, ThinkEye, sports an embedded operating system and uses a licensed image-processing library called Common Vision Blox from Stemmer Imaging GmbH of Puchheim, Germany. The library runs on either ThinkEye or a PC. It provides such capabilities as calculating the shape of an object and edge detection.
JAI's ThinkEye smart camera adds brains along with network connectivity.
ThinkEye doesn’t use a neural network. Instead, its image processing is based on the more traditional embedded reduced-instruction set computer (RISC) approach, with C++ programming and dynamic link libraries.
“It requires programming skills, but, of course, provides full flexibility to the user,” said Gunnar Jonson, director of product marketing at JAI.
ThinkEye illustrates a network connectivity trend common to many cameras. These devices host a Web server and have an Ethernet connection, so they can be programmed from or have results sent to anywhere via a standard Web browser. Thus, in a factory where parts are being inspected prior to automated assembly, an engineer working at a desk can set up the camera, and the images and outcomes can be transmitted from the camera to that same desk or to any other for further analysis.
The final development — better hardware — is a result of many enhancements and is a mixed blessing. More powerful hardware, for example, can sometimes be more complicated and require extensive software optimization. According to Pierantonio Boriero, product manager at Matrox Imaging in Dorval, Quebec, Canada, processing power is another hardware dilemma for smart cameras. With power consumption issues and heat dissipation, smart cameras cannot be fitted with the same processors as a PC, so there is a limit to the types of applications that can be run on them, he said.
A smart camera and the associated image processing allow this robot to scan and range a car chassis. Courtesy of Vision Components.
But such problems are overshadowed by what these advances make possible. Endre Toth, director of business development at Vision Components GmbH in Ettlingen, Germany, noted that advanced digital signal processors and improved systems architecture make it possible for today’s cameras to communicate over the Internet while capturing and processing images. One of his company’s latest products, the VC2048E, can find edges and shapes at a rate of up to 110 fps.
Using normalized gray-scale correlation, the camera captures a target pattern and uses it as a template. Every succeeding image will be searched for that pattern. The success of that search will depend on the quality of the initial image and that of all subsequent ones. That has implications for camera operation and setup.
As Toth noted, “First what you try to do is capture as accurate an image as possible.” No matter how powerful the hardware gets, how connected the cameras become or how clever the processing may be, the basic need for high-quality images won’t change. What will change is what can be done with that data, and that will be due to improved image processing.