Vision systems can be as important to manufacturing businesses as the manufactured products themselves. For products ranging from semiconductors to solar cells, and from pharmaceuticals to vehicles, vision systems help ensure quality and maximize production line efficiency, which means cost savings. Global sales revenue from machine vision systems in North America was valued at $889.3 million in 2009, according to the latest market study by the Automated Imaging Association (AIA). The semiconductor industry tops the list of users both in terms of revenue and units sold. This is followed by the automotive and wood industries. While vision systems have enjoyed a strong presence in these manufacturing areas for many years, they are now starting to catch the attention of nonmanufacturing enterprises. “Because of the attractiveness of the machine vision value proposition – increased efficiency, improved product quality and lower costs – machine vision technology will find success in a number of nontraditional and emerging markets,” said Paul Kellett, AIA director of market analysis. “As time goes on, I fully expect machine vision companies to increasingly target nonindustrial customers – that is, to focus beyond the factory in their sales campaigns.” From economic lows to tech highs The recent economic downturn hit the manufacturing industry hard and, as a result, the machine vision market suffered too. In fact, revenue declined by 29.4 percent from 2004 to 2009, according to AIA’s study. Even though the greater economy may be experiencing a recovery, most economists view the comeback as tentative and weak. What’s more, many feel that a double-dip recession cannot be ruled out. Despite this, Kellett is cautiously optimistic about machine vision. “While there is some evidence that the ‘Great Recession’ interrupted to some extent long-term trends (as customers bought down in response to shrunken capital budgets), I strongly suspect that the most recent sales data will show a return to the long-term trends,” he said. “Our new AIA sales tracking report shows that machine vision system sales in North America are up an average of 46 percent in the first half of 2010 over the first half of 2009.” Economic pressure aside, the vision market has historically been driven by technological trends. One such technology that has evolved noticeably in 2010, and is expected to continue into 2011, is the camera – the eyes of the vision system. The trend here is increasingly digital, faster and higher resolution. As users call for better resolution and speed, demand for greater bandwidth intensifies. As such, the migration from analog to digital and between different digital interfaces continues as users move up the “bandwidth ladder.” The adoption of the GigE Vision standard in May 2007 represented a major market milestone; in 2009, only 14.2 percent of total machine vision camera units and 23.2 percent of revenue were due to prestandard GigE cameras – those that purported to have a GigE interface but that did not officially conform to the GigE standard, since it had not yet been officially adopted. Industry experts expect the GigE Vision camera and its extension, 10 GigE, to gain an increasing share of total machine vision camera sales over time but not to supplant all other types of cameras. Another trend is the growing reliance on field-programmable gate arrays within digital cameras to perform image preprocessing tasks. Some say that this trend is eroding the distinction between the camera and a relatively new product: the smart camera. Smart cameras combine the functionality of a machine vision system with lower prices and have, not surprisingly, experienced explosive growth. Typically, a smart camera houses a camera, an image processing unit and machine vision-related programs within the same casing. When smart cameras were invented, their capabilities were somewhat limited. Their lack of tools and processing power meant that they were generally suitable only for a narrow range of simple tasks. Typically, these early smart cameras were used for detecting the presence or absence of items but were not particularly “smart” by today’s definition. Currently, smart cameras are the fastest growing sector of the machine vision market. And such products are opening the door to some of the emerging nonmanufacturing applications of vision systems, such as biometric recognition and intelligent surveillance. Smarter surveillance Video surveillance has moved on from the indiscriminate capture of images to more intelligent traffic surveillance systems, for example. Smart surveillance systems can monitor the flow of traffic, alert authorities to possible accidents and identify number plates. Perhaps the next step in intelligent surveillance is a system with a humanlike ability to learn from the environment it oversees. In the SEARISE (Smart Eyes: Attending and Recognising Instances of Salient Events) project, which comes to fruition in February 2011, a trinocular active cognitive vision system is being developed. The Smart Eyes system picks out two possible salient events during a soccer match. Waving flags (green box) are deemed not to be security relevant, whereas the person on the edge of the pitch (red box) is determined to warrant more attention. Courtesy of the SEARISE partners. Unlike other approaches in video surveillance, the Smart Eyes system will be able to learn continuously from visual input. It will self-adjust to changes in the visual environment as well as concentrate its focus on salient events. Saliency is a measure of relative novelty, so a salient event is one that stands out from its surroundings. Since not all salient events will be security relevant, the project members have developed software to help the system “learn” to focus only on security-relevant events. SEARISE software can learn from a few video samples of security-relevant events specified by experts. The software then analyzes all salient events in the scene to detect the specified security events; it can update its knowledge about the learned events via automatic online learning. Additional interactive learning allows new security-relevant events to be incorporated into the previously learned model. These events will then be categorized according to context, and the smart camera will duly assign the majority of computational resources to the informative parts of the scene. The aim of the project is to develop a prototype of Smart Eyes and put it to the test in real-life scenarios featuring the activity of people at varying distances from the camera. At a long-range distance capacity, the system will monitor crowd behavior of sports fans in a soccer arena. At short range, the system will be tasked with monitoring the behavior of small groups of people and single individuals. The technology behind Smart Eyes consists of three cameras acting in unison in a coordinated “recognition loop.” A wide-view global camera performs general monitoring and, once a particular event of interest is identified, active binocular stereo cameras zoom in for a closer look. The SEARISE Project has been supported by the European Union’s FP7 Programme and is made up of a team of academic and industrial partners across Europe. The smart vision cameras can be used for security applications in public places such as sports stadiums, city streets, metro systems and airports. Driving vision into autos Still another trend is lower cost, as subcomponents become less expensive. As a consequence, vision systems are offering increasing value and appeal in commercial markets. For example, vehicle manufacturers have discovered the benefits of thermal imaging to enhance driver vision. Of course, for many years the military and police have used thermal imaging for night vision for security and surveillance applications. In fact, in places where constant guarding is necessary, such as airports, ports and nuclear facilities, thermal imaging will often be employed. Thermal imaging specialist Flir Systems recognizes that, thanks to trends toward more compact systems, better image quality, more powerful software and falling costs, thermal imaging cameras are making their way into the mainstream. Automotive night-vision systems enable drivers to detect and monitor potential hazards on or near the road, allowing more time to react to danger. The PathFindIR thermal imager helps the user recognize pedestrians, animals or objects in total darkness, smoke, rain and snow. Images courtesy of Flir Systems. “Thermal imaging cameras are becoming more and more popular for a wide variety of applications,” said Christiaan Maras, Flir Systems’ marketing director for Europe, the Middle East and Africa. “Volumes are going up, and prices are coming down. This will help thermal imaging make its way to numerous users and applications for seeing in the dark, measuring temperatures and so on.” Flir Systems’ PathFindIR thermal cameras are already sold as an option on selected BMW series 5, 6 and 7 models for enhanced driver vision. The PathFindIR incorporates an uncooled 320 x 240-pixel microbolometer and a 19-mm wide-angle lens to provide a 36° field of view. Measuring just 5.8 x 5.7 x 7.2 cm and weighing 360 g, the compact camera is installed behind the vehicle’s grill. Arming vehicles with vision technology is becoming big business. As featured in the October 2010 issue of Photonics Spectra (“On the road with Junior: A tale of optics and driverless cars,” pp. 34-37), Volkswagen and the Stanford University School of Engineering are working together as part of the Volkswagen Automotive Innovation Laboratory (VAIL). The group is looking into ways of transforming the information from cameras to provide details of a vehicle’s surroundings in 3-D. To do this, VAIL is exploiting lidar data to better interpret the images provided by cameras. Being able to use cameras alone for accurate information about distance and shape will make it easier to integrate vision technology into commercial vehicles. Furthermore, vehicle manufacturers can take advantage of the fast frame rates, higher resolution and lower cost offered by today’s cameras. The Flir Systems PathFindIR is a compact thermal imaging camera that reduces the hazards of nighttime driving. Courtesy of Flir Systems. Lights, camera, action With the trend toward nonindustrial applications, vision systems are being taken out of the factory setting – which means that new considerations come into play. For example, outdoor environments with their lighting variables and temperature changes can pose a problem for imaging and vision manufacturers, and they take on more importance in camera performance. For John Merva of Advanced Illumination, based in Rochester, Vt., getting the correct lighting figured out is the most important thing. Advanced Illumination’s new expandable spot/linear array features high-brightness LEDs and an extruded aluminum housing with built-in mounting features and an efficient heat transfer design. Images courtesy of Advanced Illumination. “The image seen in the camera is light reflected from the object. Incorrect lighting equals a bad image,” Merva said. “Vision has moved from complex, high-cost products to all parts of the consumer product market and, as always, a proper lighting solution saves on the cost of a more expensive processing solution.” Incorrect lighting can lead to serious imaging problems such as blooming, hot spots and shadowing, which can hide important image information. What’s more, nonuniform lighting can result in a low signal-to-noise ratio. Selecting the proper illumination is no easy task. The solution often appears to be a mixture of science and art. However, Merva points out that light is governed by the laws of physics, and solutions are based on scientific principles. Leading lighting suppliers provide an application evaluation and recommendation service free of charge. To help customers make better decisions, Advanced Illumination even offers free education seminars related to the application of lighting. Advanced Illumination’s new 6-in.-diameter low-angle dark-field light. Courtesy of Advanced Illumination. Lighting vendors also face a numberof customer-related challenges since one lighting solution does not fit all applications, and customers will often require a solution in just one to two weeks. Merva advises that the best way to avoid a last-minute crisis is to seek the proper lighting solution at the inception of a project. “As I review sales of our various products, I see a clear trend toward lights that have higher optical output and more complex, purpose-driven designs,” Merva said. “These lights allow delivery of very unique geometries of intense light.” Perhaps the most obvious trend that is taking imaging and vision applications by storm is the tenacious rise of LEDs. LEDs have almost doubled their share of machine vision lighting units sold from 52.4 percent in 2005 to 99.8 percent in 2009, according to AIA’s market study. The battle of technologies has been clearly won by LEDs, which have supplanted all other lighting options, except in special cases, the report details. AIA’s latest annual study also warns that, with the LED now the undisputed king, the overall strength of the machine vision lighting market will rest primarily on the average price and number of LEDs sold. If the average price of LEDs continues to drop, suppliers must sell an increasing number of units to offset the impact on total market revenue. But it’s not just the price of LEDs that lighting vendors must adapt to; LED sources have evolved to allow high-power output in a variety of wavelengths. This increase in optical output contributes to high LED die temperatures, which is one of the more significant factors that leads to early failure of the LED. “Cold environs are not a problem for LED light sources, as they only assist in heat removal. However, hotter temperatures do provide a need for additional heat transfer measures,” Merva said. “Advanced Illumination has paid considerable attention to LED die temperature and effective heat transfer. The results of this can be observed in our product designs which incorporate extensive heat transfer features. This provides customers with LED illumination products that can operate for tens of thousands of hours in hotter environments.” Merva also notices a trend toward applications that employ line-scan cameras for inspection of textiles, films, semiconductors, lumber, roads and rails. Line configurations represent the most popular lighting geometry at 29 percent of the units sold and 29.3 percent of the total sales revenue, AIA reports. Advanced Illumination’s newest high-intensity line light for line-scan applications. Also available as a backlight, it features remotely adjustable intensity with onboard current drive and passive cooling. Available in expandable sizes up to 96 in. Courtesy of Advanced Illumination. In comparison, area lighting represents 15.1 percent of the units sold and 15.3 percent of the revenue. This is followed by backlighting, which comprises 10.2 percent of units sold and 13.7 percent of total revenue. Computer vision: There’s an app for that Vision technology is not just about improving image capture; there is also a lot happening behind the camera in terms of image processing. Computer vision may be in its infancy, but smart phone users are already taking advantage of advances in this field, thanks to a new application from Google. Google Goggles at work identifying Jackson Pollock’s Convergence as well as the cover of Jack Kerouac’s On the Road. Images courtesy of Google. We are all familiar with searching the Internet for information by typing words into a browser or, more recently, speaking phrases into a smart phone. Google Goggles has provided Android users with a visual search tool since December 2009, which was then released to iPhone users in September 2010. By simply capturing an image of an object of interest, Google will analyze the photo for a few seconds and then offer some relevant search results. The service is in its infancy, and it currently works best with common landmarks, businesses, bar codes and products such as books, movies and wine bottles. However, Google hopes to increase the number of objects that can be recognized and provide better and more relevant responses when there is a match. “Goggles is a result of computer vision research work done by our team over many years,” said Shailesh Nalawadi, a Google Goggles product manager. “We felt it was only natural that users should also be able to search for information by pointing to the object of interest using their smart phones to visually capture information. Over time, we feel this can become a very natural way for users to find out about items in their physical surroundings.” The Goggles team has created a database of recognized objects and associated metadata about the objects. Each recognized object is converted to its mathematical representation and stored in a server. When a user selects Goggles to capture a query image, the image is compressed on the smart phone and transmitted to Google’s recognition servers in the cloud. The recognition servers then convert the query image to a mathematical representation, which is compared with all previously recognized objects in the database in a matter of seconds. “We compile all the matched images and the associated keywords and stream it back to the user’s handset,” Nalawadi said. “The user can select the result that most closely matches the query, and we render search results for the match.” If computers can be taught to reliably “see,” Nalawadi predicts a revolution in the way humans interact with the world around them. “Imagine walking through a forest while having access to information about all the plants and animals around you. Or getting your computer to suggest the next move in a chess game, or being able to pull up product information by pointing the phone at a cereal box in the grocery store aisle,” he said. “The possibilities are immense.”