Smart Cameras Are Earning a Niche in Machine Vision

Hank Hogan, Contributing Editor

With apologies to Forrest Gump: Smart is as smart does. As the cost of memory and processor chips has dropped, camera vendors have crammed more and more intelligence into their devices. Consequently, current instruments can run the latest operating systems and software, enabling users to shift from desktop use directly to smart-camera operation without suffering any loss of performance.

A sample of the new breed of instruments is the XCI-SX1 Smart Camera from Sony Electronics Inc. of Park Ridge, N.J., which fits in the palm of one’s hand yet contains a complete vision system: sensor, processor and interface. The device has 256 MB of fast memory, 1 GB of flash memory (which acts similarly to a PC hard drive) and XPe, an embedded version of Microsoft’s XP operating system.

Smart cameras, such as the one pictured from Sony Corp., combine an imaging chip, a lens, a PC and an interface inside a single package, enabling ease of setup and use in a variety of applications. Image-processing software can be loaded directly into these systems, and captured images can be processed by the built-in computer or delivered off-site. Courtesy of Sony Corp.

“It is a mainstream PC running XPe and using an x86 architectural processor,” said Ilias Levis, product manager for visual imaging products at Sony. That structure, he said, is in contrast to some other designs, which use a custom processor or unconventional operating system.

Furthermore, more than 70 percent of the flash memory is available for data storage and applications. Sony hopes the latter will be easier to implement given the demonstration version of Vision Builder AI software from National Instruments of Austin, Texas, that ships with the camera. According to Levis, software for the system also is available from Dalsa Coreco of St. Laurent, Quebec, Canada, and from MVTech Software GmbH of Munich, Germany.

Although Sony’s package illustrates recent technological advances and shows that good things do come in small packages, barriers loom to further improvements to smart cameras, including the need to dissipate heat and the desire to hold down costs. A look at innovations and market trends shows where the technology is heading and provides an answer to the question of just how smart a smart camera is.

Cameras or sensors?

Not all companies call their vision products “smart cameras,” even if they largely are functionally equivalent. Some products are dubbed “smart sensors,” “intelligent cameras,” “embedded vision processors” or “compact vision systems.” All integrate some or all of the basic building blocks of machine vision into a small package, acquiring images, extracting data and processing information with embedded computing power. They perform these tasks with varying degrees of flexibility and differing capabilities.

Paul Kellett, director of market analysis for the Automated Imaging Association of Ann Arbor, Mich., said that the differences between the various designations are not always well understood. “The distinction between them that is perceived in the market is fuzzy at best,” he said.

According to the Automated Imaging Association (AIA), the North American machine vision smart-camera market grew from $32.5 million to $99.2 million between 2001 and 2005, an annual growth rate of more than 32 percent. Courtesy of the AIA.

The association estimates that the North American machine vision smart-camera market, using the term broadly, grew from $32.5 million in 2001 to $99.2 million in 2005, a compound annual growth rate of more than 32 percent. Smart cameras have not consistently become less expensive over that time, according to the association’s numbers, although Kellett noted that the lower end of the market has dropped in price. That might be due to competition among the 30 or so vendors, because they offer a variety of products for different situations and, in some cases, for less-demanding applications.

Vision sensors locate, identify and inspect parts on a conveyor during an automated assembly process. Courtesy of Cognex Inc.

‘Vision sensors’

Cognex Corp. of Natick, Mass., claims the title of largest smart-camera supplier. It prefers to call its products “vision sensors” to emphasize that they are an integrated unit that is applied like an industrial sensor, said Bryan Boatner, product marketing manager for the company’s In-Sight line. The term also distinguishes the product from a PC-based machine vision system, which is more expensive and more capable. In-Sight systems have to be configured while connected to a PC but, after that, they detect defects, recognize or verify printed characters, locate patterns or perform other tasks on their own.

Increasing computational power within its vision sensor products has enabled Cognex to implement more sophisticated pattern-matching algorithms, Boatner said. As a result, the recognition speed, accuracy and flexibility have improved, allowing the devices to be used in vision-guided robotics.

In the past, robotic applications used precision fixtures to ensure that a workpiece always was presented in the same way. Using machine vision equipment instead of a fixture provides two benefits, according to Boatner. Smart cameras typically are less expensive than precision fixtures. In addition, they allow mixed model production or flexible manufacturing because there is no need to change fixtures when changing parts. Instead, a controller tells the sensor that the line is running part B instead of part A. “It changes on the fly with no time and no labor required,” he said.

A vision sensor provides location data to a Mitsubishi Electric Corp. robot that is picking unfixtured products from a conveyor and placing them into cartons. Pattern matching allows the sensors to relocate objects accurately, even when they overlap. Courtesy of Cognex Inc.

At the other end of the size scale, FastVision LLC of Nashua, N.H., is a privately held start-up company that is taking a different approach to smart cameras. FastVision’s CEO, Joseph Sgro, stated that the company emphasizes speed above everything else, which has implications for the sensor at the heart of its smart cameras: The company has chosen to implement CMOS imagers instead of the more traditional CCDs.

“The advantage of CMOS in the market is that it is very fast,” said Sgro, who acknowledged that the picture quality achieved with CMOS might not be as good as that from a CCD. He added, however, that the company’s 1.3-megapixel cameras can acquire 500 to 1000 frames per second and its 4-megapixel cameras, up to 200 frames per second.

With embedded processing, a smart camera can reduce the torrent of data issuing from a fast vision sensor to a more manageable flow. Courtesy of FastVision LLC.

With that much data being acquired, processing must be done using a field-programmable gate array or a fast processor if the camera is used for real-time processing or compression. Without onboard speedy computation abilities, the camera can act only as an event recorder, with analysis done later.

FastVision’s products all have field-programmable gate arrays, which present challenges in programming. However, Sgro said that the company will soon release processor-based offerings that could be used for applications where relatively rare events must be detected among a great many normal ones — for example, during inspection of LCD panels for scarce catastrophic defects or for classifying cells or other objects.

Kerry van Iseghem, the founder of Imaging Solutions Group of Fairport, N.Y., noted that his company — another small smart-camera business — is “sensor agnostic,” using CMOS or CCD imagers as the application dictates. The company provides custom camera system design along with a smart-camera product line. Most of its products — either custom or standard — implement proprietary image-processing algorithms. To maximize computational speed, this is done by using a field-programmable gate array or an application-specific integrated circuit. Overall, he added, moving functions from the PC to the camera provides significant benefits.

“You can integrate vision into smaller and smaller applications or physical locations than you could do before. It enables people to lower their cost dramatically,” he said.

Van Iseghem gave as an example a custom project that replaced a system consisting of a high-performance PC, multiple cameras and matching acquisition boards. The smart-camera solution comprised a single small processor board, along with one daughter card per camera. The result was a savings of thousands of dollars per system.

Bumping up against limits

Vision Components GmbH of Ettlingen, Germany, offered one of the first commercial smart cameras, running a 16-bit processor at about 30 MHz. Its latest products sport a 64-bit architecture and 1-GHz processors. For optimal image quality, all of the cameras have CCD sensors, according to director of business development Endre Toth.

He noted that smart cameras have benefited from continuing advances in semiconductors, as evidenced by the improved speed and other changes in the company’s product history. However, as with many other manufactured goods, smart cameras have much longer product life cycles than do consumer electronics. For example, PCs and cell phones — two driving forces in the consumer market — become obsolete and are usually replaced within a few years. Industrial products, on the other hand, typically have life spans measured in decades.

Toth said that the semiconductor components could become difficult to get long before product lifetime expires. For example, many older chips are manufactured with lead-based processing, something that is en route to being banned worldwide. For the most part, pin-compatible replacement chips are, or will be, available, but that is not guaranteed. For that reason, the electronics that make smart cameras smart may effectively disappear.

“Sooner or later, you are not going to be able to find the old components which contain lead,” Toth said.

Vision Components already is working on this issue. The example, however, illustrates what can happen to an industrial product.

Another problem that is confronting smart-camera manufacturers is effectively dispelling the heat generated by the electronics that are crammed into a small enclosure.

“That is, more or less, the limiting factor for the next step of performance,” said Werner Borchert, product manager for intelligent camera products at Basler AG of Ahrensburg, Germany.

The problem is worsened because smart cameras — by their nature and because of where they operate — cannot be equipped with cooling fans. They also have to be ready to respond to an external trigger, which could come at any time and might signal a fleeting event.

According to Borchert, Basler employs ultralow-voltage electronics, including processors to address the issue. Other aspects of the company’s approach to the problem include using highly integrated processor devices, engineering of enclosures, and housing variants to give customers flexibility with regard to integration while ensuring sufficient heat dissipation. One trick used in laptops — a sleep mode that lowers power consumption — is not feasible because there typically is not enough downtime in the production cycle.

Is playing dumb smart?

The dividing line between smart cameras and their so-called “dumb” counterparts has become increasingly blurred. Steve Kinney, product manager of JAI-Pulnix Inc. of San Jose, Calif., noted a critical difference between the two types: Smart cameras demand a mix of expertises in software, hardware and applications that traditionally have been the domain of vision integrators. It was the difficulty of balancing these elements while hitting profit-margin goals that led JAI-Pulnix to exit the smart-camera business a few years ago.

Kinney said that the company, in a sense, never really left the smart-camera market, however, and, in fact, may have expanded its product line. “Nearly every camera we make today is a smart camera in one way or another,” he said.

He explained further that inside most cameras sits a field-programmable gate array, an application-specific integrated circuit or some processor. Many cameras also come equipped with memory and, if not, it is easy to add. The result is that even cameras that are not designated “smart” can do some processing. For example, such a system can inspect bottles for the presence or absence of a blue sticker. It might even be up to the more complicated task of reading a license plate as a car speeds past a tollbooth.

The key distinction is that these devices are neither general-purpose systems nor even ones with limited flexibility. End users cannot change any settings or parameters, which makes support easier and less expensive, and the cameras are capable of some amazing feats, making them the imaging equivalent of idiot savants and, in their own way, the Forrest Gumps of the camera world.