Autonomous cars and 3D imaging robots are driving demand for CMOS sensors.
MARK BUTLER, TELEDYNE DALSA INC.
It is no secret that CMOS image sensor technology has undergone a rapid transformation. Driven by the consumer cellphone market’s insatiable demand for fast, high-quality images, there has been substantial investment in CMOS sensor technology, resulting in products that deliver smaller pixels, higher sensitivity and lower noise at a decreased cost.
This intense pressure from the consumer market has benefited other market segments, providing cheaper, more powerful image sensors and expanding possible applications including machine vision, medical imaging and security. Going forward, these less-expensive, more powerful sensors also present big opportunities for emerging applications.
Revenue growth in CMOS sensors by vertical industry, from 2015 to 2020. Data from IC Insights OSD as reported in Image Sensors World. Courtesy of Teledyne Dalsa.
Market watchers are forecasting continued, explosive growth in the CMOS image sensor business. This predicted massive growth will be driven by some exciting market segments.
The automotive industry has the greatest potential, by far, with forecasts estimating that automotive applications will drive an increase of $1.83 billion in CMOS sensor revenue from 2015 to 2020 according to a presentation by Mentor Graphics with data from IC Insights. Other market segments expected to drive increased demand for CMOS sensors in the next five years include medical and scientific imaging with $710 million revenue growth, security also with $710 million revenue growth and industrial applications with $520 million revenue growth, also according to IC Insights OSD-2016 data.
Consumer smartphones, which all have integrated cameras, will generate the highest volume of image sensors, but as a decreasing segment of total CMOS image sensor sales. Camera phones — with 70 percent of the total $9.9 billion CMOS image sensors market in 2015 — are predicted to decrease to 48 percent of the forecasted $15.2 billion market in 2020, whereas the automotive industry is truly the standout for growth potential as it grows from 3 to 14 percent of total CMOS sensor revenue.
Autonomous cars spur demand
The push to develop and deploy autonomous, or self-driving, cars is underway. Plenty of heavy hitters like Google and Tesla, not to mention dozens of auto manufacturers, are trying to establish themselves as the leader in the emerging technology. Critical to the development of autonomous vehicles is the capture, storage and processing of enormous amounts of visual information in and around the vehicle. This essential element will drive demand for image sensors in the automotive industry going forward.
Even without the promise of autonomous vehicles, some high-end cars on the road today already have as many as 20 image sensors incorporated in the vehicle for existing safety technologies like back-up cameras, blind spot warning systems, pedestrian detection and lane departure warning systems. These technologies will continue to spread throughout the auto industry, along with other new features that promise safety and assist navigation, even as the self-driving car revolution takes hold.
Medical imaging is contributing to a $710 million growth between 2015 and 2020. Courtesy of Teledyne Dalsa.
Imaging has been — and continues to be — critical in the medical industry. Three-dimensional imaging is finding expanded use in mammography and other types of x-ray. As the population ages, and the need for more advanced diagnostics grows, CMOS sensor manufacturers will continue to see more opportunities from the medical industry.
As the population grows globally, demand for CMOS imagers for food-related applications will expand in order to optimize food production from limited agricultural space and to ensure the safety of our food. A newer application is aerial imaging of agriculture fields to optimize soil nutrients and monitor soil moisture — even linking agricultural imaging to the Internet of Things (IoT). A more established application is food inspection via imaging. Today’s inspection systems can remove unwanted contaminants, such as small pebbles in grain or rice, and can determine whether food is bruised or contains unwanted bacteria — all enabled through CMOS imagers.
Security and surveillance cameras are already ubiquitous, and the next five years will continue the trend of sizable growth in these applications as cheaper, more powerful sensors enable smaller, smarter cameras that can be deployed in a variety of conditions.
Thermal imaging continues to drive sensor growth. Above is an IR image of a car engine (a), vacuum pump (b) and building (c). Courtesy of Teledyne Dalsa.
The evolution of thermal imaging is one application that will drive sensor growth for the security industry. Thermal imaging technology can enable surveillance that monitors the movement of people in areas that are dark or difficult to see in. Surveillance cameras using thermal imaging can “follow” someone trying to stay hidden in a dark area, and was notably used by police to find and capture one of the Boston Marathon bombers who was hiding in a boat stored in someone’s backyard.
Industrial robots
Advances in 3D imaging will continue to drive advances in manufacturing including robotics technology that enables machines to do many repetitive tasks now done by humans. Three-dimensional imaging is helping robots use position recognition to master the task of “bin-picking” — identifying an object and picking it out of a pile. Although it seems simple, this monotonous and labor-intensive task relies on human vision to discriminate one part from another since parts are all lumped together without uniform shapes or orientation. As robotics in manufacturing become more sophisticated, this type of application will become more widespread.
Aided by a 3D area sensor to locate random parts in a bin, a robot plans its next pick. Courtesy of Motion Controls Robotics.
In fact, we are already seeing a combination of robotics and self-driving technology in some controlled manufacturing environments, where a robot can self-navigate through a plant, pick up a large pallet of material and move it to another part of the building. Expect to see these kinds of applications growing explosively at industrial sites as the technology simultaneously becomes more sophisticated and less expensive.
The new applications driving growth in imaging products rely on nontraditional imaging modalities including 3D vision, thermal infrared imaging and sensitivity beyond the visible. Investments in these modalities over the next few years could rival advances made in the last 15 years in smartphone image capture. These new imaging modalities promise to become the game-changers that drive demand for smaller, cheaper, more powerful CMOS sensors going forward.
Three-dimensional vision is especially interesting because there are a number of competing 3D technologies at work, so there is some uncertainty as to which technologies will thrive and how many different technologies will be needed to deliver 3D vision applications across various industries.
For instance, there are at least three contending optical technologies competing for automotive applications: time-of-flight (ToF), which is essentially radar with light, where a pulsed or time-modulated light source is used to measure distance based on transit time and speed of light; binocular vision, which enables 3D vision via two sensors at a set distance apart on the camera; and structured light, which is a 3D technology that projects a pattern onto an object and then detects spatial shifts to determine how far away the object is. The use of these three technologies depends somewhat on the length scale in the applications: Is the application measuring submeter, meter or hundreds-of-meters distance? Because of these differences, it is unlikely that one specific 3D technology will claim the automotive market. Beyond the automotive market, there are even more 3D technologies, such as sheet of light, which is used in many manufacturing applications. With competing technologies and applications that stretch across industries, it will be interesting to watch the continued evolution of 3D imaging.
The other big area for growth in 3D imaging will be in robotics. Today, manufacturers are harnessing the power of 3D imaging to enable robots to “see” as a person does. This “vision” enables robots to do tasks that were previously too difficult. Robotics will continue to see wide adoption in manufacturing, with robots doing many tasks that humans can do, and even a few that humans cannot do.v
Service robots
In addition to industrial robotics in manufacturing, advances in sensor technology will help develop 3D vision that can enable service robots for at-home applications. Beyond today’s robot vacuums, we may eventually see robots that can mow the lawn and clean gutters, as well as provide cognitive support as human companion robots and complete tasks for the elderly or disabled. Service robots are finding more and more uses in offices, hospitals, museums and other public places, too. Some examples of popular applications are self-navigating lawn mowers, window washers and surgical operators. While growth will primarily be driven by their use in homes, service robots are also being used in agriculture, forestry, mining, cleaning, construction, demolition, freight transport, and even search and rescue. Asia-Pacific is currently the largest and fastest growing market.v
Similar to service robots, collaborative robots (or cobots) work together with people and have built-in abnormal force detection to ensure no harm comes to the individuals they work with. Expected areas of growth include the food industry, agriculture and cosmetics.
Seeing beyond the visible
Another area of technology development being driven by high-volume applications is thermal infrared imaging, referenced above in our discussion of security and surveillance applications. In these types of applications, thermal imaging allows us to see objects that are not visible to the human eye by detecting the heat emitted by these objects.
Such technology is already available in some high-end car models to assist in the detection of pedestrians or animals in the road that might otherwise be obscured by fog or darkness. As sensor costs shrink, the cost of this technology will also drop, and we will find it in most future vehicle models. The cost reduction is likely to come again from pressure in the consumer smartphone market, anticipating consumer demand for mobile apps that provide thermal imaging for security purposes, such as to detect a person lurking in or near a car parked in a dark area.v
Spectroscopic imaging is another candidate for high-volume applications, though it is not yet ready for widespread adoption. However, as spectroscopic imaging develops for these applications, it could also have positive implications for machine vision. Spectroscopic image sensors gather information about the materials being imaged by detecting the spectral signatures associated with those materials — the fingerprint of each component captured by the peaks and valleys in a material’s absorption and reflection spectra. This method could be used to detect substances that would not otherwise be distinguishable by the human eye, such as a black grain or rock mixed in with tea leaves.
One commonly cited high-volume application for this would be a smartphone application that would use spectroscopic image sensors to scan a consumer’s dinner plate in a restaurant to check for signs of foodborne illnesses. The growth of such high-volume applications for spectroscopic imaging would lead to lower sensor costs and more compact technology, again benefitting machine vision. Lower costs could lead to the use of spectroscopic technology in machine vision applications for industrial process control — for example, to aid in the detection of contaminants. Spectral imaging could also help with the detection of counterfeit products if manufacturers were to embed markers with specific spectral signatures into products that were not visible to the human eye, but were able to be scanned to confirm authenticity.
There is no doubt that the CMOS sensor market is poised to deliver exciting advancements in sensor technologies and to realize growth in exciting markets like automotive, medical, security and machine vision. As consumer technology demands drive sensor advances and reduced sensor costs, look for these advances to ripple through business-related applications as well.
Meet the author
Mark Butler is group manager of product management and marketing at Teledyne Dalsa Inc. He holds a Bachelor of Science in electrical engineering from the University of Waterloo and an MBA in strategy, marketing and finance from Wilfrid Laurier University; email: [email protected].