Robert Howison and Bruno Ménard Teledyne Dalsa
Color has been considered expensive in terms of money and processing power, but these costs have decreased. Here’s how color vision works and why it can be beneficial for today’s applications.
Vision has evolved into a fast and reliable tool for applications such as quality inspection, traffic surveillance and target tracking. In many cases, a vision system can perform optical tasks more quickly and accurately than humans – and at a lower cost. But how does introducing color into the equation help improve the quality of the results?
A vision system acquires images of an object with a camera and uses computers to process, analyze and measure various characteristics of that object – such as color – so decisions can be made.
In the past, color was not widely used in vision systems because the costs and processing power requirements were high. However, as prices have decreased and power has ceased to be an issue, solution providers have begun to consider incorporating color into vision systems for a higher-quality result.
By the end of 2011, color cameras comprised about 20 percent of cameras sold for imaging applications, according to market studies by the Automated Imaging Association (AIA), the industry’s trade organization. Most experts believe that the use of color will expand in the vision domain. Color provides much more visual detail than monochrome gray scale and adds a new dimension to real-world data analysis. For example, color increasingly is being adopted in bank note inspection applications for scanning and processing.
In China, Japan and South Korea, color inspection is required by the government because people use a personal seal rather than a signature in issuing personal checks. Another example is traffic surveillance: Color cameras can help identify key information, such as traffic light color or a picture on the background of a license plate.
Color is better, right?
The basic assumption is that color is more “advanced” than black and white, or monochrome, so it must be better; however, in the case of vision, this is not always true.
In typical applications, such as detecting cracks, scratches or other defects, color is not absolutely necessary; the goal is to discern a difference in lightness on the object’s surface. For some tasks, monochrome cameras are even better than color. In many other cases, color images do not offer a significant advantage over monochrome in resolving a vision problem. Examples are optical character recognition or verification, bar-code reading, gauging and other applications that rely mainly on high-resolution “spatial” information.
Industrial quality control applications such as paint inspection can benefit from color vision.
Of course, a machine cannot actually see in color. Machines use mathematical models to approximate human color detection. A machine can be calibrated against the average human response to colors and, hence, “see” them – meaning that it gives consistent responses to colors observed in a controlled setting. This “calibrated color vision” is useful for measuring and matching colorants in paint, plastics and fabrics. We don’t think of this as seeing color the way a human does, except perhaps as a philosophical exercise. However, it is important to make the distinction between the relative measurements that can be made with a vision system and the absolute measurements that are possible only with devices such as photospectrometers.
Human color vision evolved to extract consistent information about the material properties of objects seen under huge variations of illumination and view. For example, fruit color must be reliably determined in varying lighting conditions so that we can distinguish ripe from unripe or bad fruit. No one wants to eat a blotchy orange or a half-yellow lime.
Industrial quality control applications such as printing inspection can benefit from color vision.
Human color vision thus has mechanisms to factor out variations in illumination and view that we cannot – or do not bother to – incorporate into machine color vision.
Also, our color vision is relative. Because nearby colors influence our perception of color, and because there are many differences among individuals, human color vision is not a good measuring tool. It also has low resolution, a fact used to transmit color in television with very little bandwidth.
Color machine vision, on the other hand, is a better measuring tool because it is not influenced by nearby colors, and it is high resolution and does not vary much from machine to machine.
Machine vision is a reliable tool for traffic surveillance.
Unlike human vision, however, color machine vision does not have color constancy – the ability to perceive the same color independently of the light source. Systems come equipped with a color correction feature composed of precalibrated color coefficients for operation under different types of light sources (LED, halogen, incandescent, daylight, etc.) and varying color temperatures. With the appropriate correction, color cameras can output values such that colors are perceived as in real life.
For added precision, color may be calibrated in the field using what is commonly known as a Gretag Macbeth color chart. This ensures an accurate measure of the camera’s output under a specific light source.
Types of color vision systems
Most color vision systems use a mix of hardware and software to detect colors. For point or spot color measures, a mostly hardware solution is fine. Sophisticated detection systems rely more heavily on software to give flexibility to both designers and users. The main types of color cameras used in vision applications are 3-CCD, trilinear and color filter array (CFA), such as a Bayer pattern.
A 3-CCD has excellent color registration and can be used in the majority of applications; however, because of the design, its cost is high. Trilinear provides high performance and has the advantage of a lower cost. With its very high resolution (up to 16,000 pixels wide), it can be used in many precision applications, such as flat surface inspection. However, in certain applications involving rotating or randomly moving objects, spatial correction cannot be done properly.
CFA cameras round out the mix, offering the lowest-price solution. They tend to be used in lower-end applications and to have reduced color precision compared with 3-CCD and trilinear cameras, but there is broad understanding of the CFA patterns, and many algorithms exist to optimize their color performance. CFA and 3-CCD sensors can be either area scan (2-D) or line scan (1-D); trilinear sensors can be only line scan.
In a 3-CCD camera, color is selected using a prism-based interference filter that splits the incoming light into red, green and blue (RGB) primary components. Each primary color is detected by its respective CCD, and the final color image is reconstructed by combining the outputs from the three CCDs. All three color images are simultaneously captured at the same object spot.
In a trilinear color camera, three linear arrays are fabricated on a single die and coated with RGB color filters, respectively. These are absorbing filters using dye or pigment. The arrays detect a slightly different field of view of the object; spatial correction is needed to reconstruct an image.
In a CFA color camera, the sensor is coated with an RGB filter as in a trilinear camera, but the filter is composed of a pattern of alternating colors (one single color per pixel).
Teledyne Dalsa offers the Genie, Falcon, Piranha and Spyder color cameras, each based on either CFA or trilinear technology. As an example, the Spyder line-scan camera supports a type of CFA sensor called “dual line,” which provides better quality on the green channel as well as reduced pixel crosstalk; i.e., low neighbor-color interference.
When to use color
The most common machine vision application requiring color remains food inspection; e.g., fruit sorting. Others include measuring the concentration of chemicals or verifying part selection for automotive or electronics assembly. Even though in many applications using color is not necessary, it can sometimes help make a vision task easier by identifying objects, such as the fuse values in a car fuse box.
Should color eventually be used in all vision applications? Not necessarily. Monochrome cameras will always do some things better than color cameras. When faced with an application that raises some doubt, ask yourself the following questions:
1. Is measuring the color of objects a key factor in the overall quality of the product?
For example, traffic light surveillance is inherently related to color measurement.
2. Can color facilitate the detection or separation of objects?
A simple monochrome image sometimes leads to ambiguity in the lightness of objects, making them indistinguishable from one another.
3. Can color help increase the relative quality of the product?
Certain vision tasks are well achieved using monochrome images, but could be improved by adding color; i.e., the success rate would be closer to 100 percent. As an example, Pharmacode bar codes optionally provide color bars.
If the answer to any question is yes, the color side of machine vision could be a big help.
A few real-world applications will help demonstrate whether color is necessary.
As mentioned above, food is probably the one application area that we understand implicitly as daily consumers of food, judging quality and consistency.
For fruit, color vision allows determination of ripeness and quality. In the case of grains and legumes, color also distinguishes foreign matter in a steady stream of product. In meat processing, color can detect spoilage and discriminate fat, bone and gristle for automatic trimming. Color vision can even inspect the “build” quality of frozen pizza. In a monochrome image, one might see whether the density of ingredients is correct; that is, whether the quantity and distribution are adequate. However, in a monochrome image, it is difficult to tell the difference between, for example, chopped ingredients such as red, orange and green peppers.
Color vision also is used in intelligent transport systems. Applications such as license plate recognition, speed and red light enforcement, toll management, traffic control/monitoring and even rail car inspection are more and more commonplace. In many uses, color is used for visualization, while processing algorithms are applied to the monochrome component only. In license plate recognition, the plate number is identified using monochrome, while for legal purposes, such as bringing evidence to court, the storage of color images is mandatory.
Domains that traditionally have been adequately served by black-and-white vision, such as automotive, printed circuit board and print inspection, also might benefit from color for better quality.
Meet the authors
Robert Howison is OEM custom products manager at Teledyne Dalsa in Saint-Laurent, Quebec, Canada; email: [email protected]. Bruno Ménard is image processing group leader, also in Saint-Laurent; email: [email protected].