Industrial robots revolutionized manufacturing, and service robots are set to rise up and do the same in such diverse areas as medical care, surveillance and the military – and also in domestic settings, where they could take over chores from cleaning to carrying and even playing.
Recent growth in machine vision sales is undoubtedly due in part to increased investment in industrial robots, which many manufacturers see as important tools for increasing efficiency, improving product quality and lowering costs in the wake of the financial crisis. Sales continued to grow by double digits for a fifth consecutive quarter (34 percent growth in North America during the first quarter of 2011), according to a recent quarterly report from the Automated Imaging Association (AIA).
And future market penetration is expected to come from a sector that Paul Kellett, the AIA’s director of market analysis, describes as still in its infancy – the service robot.
Annual sales of professional service robots are about 13,000 to 15,000 units per year, valued at about $3 billion, according to the International Federation of Robotics (IFR) in Frankfurt, Germany. This excludes the sale of about 1.5 to 2 million units of personal, domestic service robots such as vacuum cleaners, lawn mowers and toy robots. The value of these low-cost robots is estimated at about $0.5 billion per year, and that demand is increasing.
Some experts believe that service robots will change the world the same way the personal computer did 30 years ago. It’s a bold prediction, but as Gudrun Litzenberger, general secretary for the IFR statistical department, points out, “Who would not want a robot at home taking care of tedious everyday tasks such as floor cleaning, watering flowers, laying the table or cleaning the kitchen?”
The field is relatively new, so the definition of “service robot” is still up for discussion, but it is generally considered to be a robot that is not installed in an industrial setting. It is often mobile in nature – not mounted to fixed structures as in industrial settings – and frequently requires machine vision that can handle variable outdoor environments rather than controlled industrial environments. A service robot performs tasks that are beneficial to the well-being of humans and equipment, excluding manufacturing operations.
In general, these helpful robots will turn up in either of two key areas: the professional or home/domestic domain.
Service robots for professional use are extremely diverse, since they are usually designed to perform a specific task. Some can be found in hospitals – the da Vinci surgical system developed by Intuitive Surgical Inc. of Sunnyvale, Calif., for example. This system provides assistance to surgeons rather than replacing them and is used to perform minimally invasive operations. Two high-definition cameras provide the surgeon with depth perception; a custom dual-lens endoscope coupled with two three-chip cameras effectively takes the surgeon “inside” the patient. The high-resolution 3-D endoscope has two independent vision channels linked to two high-resolution progressive-scan color monitors.
(Left) The da Vinci surgical system, developed by Intuitive Surgical Inc. of Sunnyvale, Calif., is an example of a nonindustrial service robot already in professional use. It incorporates vision technologies including high-definition cameras, a custom dual-lens endoscope, two three-chip cameras and two independent vision channels. Courtesy of ©[2010] Intuitive Surgical Inc. (Right) Doctors use the da Vinci surgical system on a patient. Courtesy of Dr. Hiep Nguyen.
Professional service robots also can be found in public buildings for delivering goods, executing tasks in dangerous or hazardous environments, or even helping in cowsheds for automatic milking, as demonstrated by LMI Technologies Inc. of Delta, British Columbia, Canada. For this purpose, time-of-flight sensors send out pulses of infrared light to determine the surface distance, helping the suction device to attach properly to the cow’s udders.
Surveillance and security sectors are also major users of service robots: They can be used to defuse mines or to operate unmanned aerial and ground-based vehicles.
Examples of home robots are vacuum cleaners and floor washers, such as the Roomba and Scooba, respectively, developed by iRobot of Bedford, Mass. Other examples of personal robots are lawn mowers, mobile media terminals and surveillance robots.
An example of a personal service robot now on the market is the Roomba 780, which uses an optical sensor to detect large and soft debris, such as popcorn. Courtesy of iRobot.
Machine vision challenge
Machine vision is a central technology for service robots, and while some of this expertise can be borrowed from industrial robots, there is much more to consider once a robot is set free from the factory and let loose into unknown terrain, surrounded by unfamiliar and possibly moving objects. Vision systems must accommodate uncontrolled environments, and designers must make sure that robots can navigate autonomously and safely – especially where people are involved.
New sensors and processes that provide real-time environment monitoring in 3-D must not only prevent collisions during manipulation but also identify and learn new objects. The increasing use of service robots in everyday environments ultimately calls for new user interfaces that can take into account the full range of communications channels.
Machine vision for service robots is closely linked to many other research areas, such as navigation, manipulation in dynamic environments, and safe human-robot interaction, said Jan Fischer of the Fraunhofer Institute for Manufacturing Engineering and Automation (IPA) in Stuttgart, Germany.
“The most prominent application of machine vision for service robots is currently the detection and localization of objects,” he said. “Especially for service robots with manipulation abilities, it is important to not only detect objects, but also determine their pose (position and orientation) in order to enable the robot to grasp and manipulate them.”
To this end, Fraunhofer IPA has developed a number of new vision technologies, including the framework for adaptive object detection and classification (differentiating whether an object is in the cup or plate class, for example). The institute has also designed sensor fusion technologies that combine stereo vision with time-of-flight sensors.
Objects are represented by so-called 6-D feature-point clouds that denote a set of 3-D points that can be assigned to certain recognizable patterns in color and depth images. By combining the color and depth images, it is possible for each feature point detected in the color image to be associated with a unique 3-D position and orientation.
To define the positions of the feature points assigned to an object with respect to each other, a local coordinate system is computed for each feature point to describe its position relative to a coordinate system located centrally in the object.
“The biggest challenge is to cope with the dynamic and unpredictable environment of service robots,” Fischer said. “Compared to industrial robots, the environment is totally undefined – e.g., lighting conditions vary, and objects appear at arbitrary poses. This is the case especially [when] objects [that] are covered and only partly visible must be handled.”
Such information processing is demanding. For object detection, which can encompass a huge variety of items, the applied algorithms must be resource-efficient – meaning that they must use a minimal amount of memory per object and detect a single object in reasonable time.
Another challenge is training the robot to recognize new objects quickly because it is not possible to teach the robot all objects in advance. The recognition of object classes is particularly challenging because each individual class has varying characteristics; the robot must identify features that distinguish between classes while at the same time capturing the commonality of a single class.
One embodiment of all of these technologies is Fraunhofer IPA’s mobile robot assistant Care-O-bot (see www.care-o-bot.de), an “interactive butler” already able to move safely among humans, detect and grasp typical household objects, and safely exchange them with humans. The perception software for the robot was developed and used within several European Union- and nationally funded projects and has already been transferred successfully to several industrial applications.
The robotic home assistant Care-O-bot 3 from Fraunhofer IPA can help in a domestic
environment. Courtesy of photographer Bernd Müller.
A sci-fi future?
Could we be in for an Isaac Asimov-inspired future? How about a sci-fi present? Different service robots designed for specific tasks are now on the market for professional and domestic applications – the Roomba, the da Vinci and so on.
IFR’s Litzenberger believes that professional service robots will move into almost all factories as logistics systems to transport goods and people; as devices for inspection and maintenance to secure operation and productivity of expensive equipment; as security systems to assist human guards; and as cleaning systems.
And we can’t forget the parallel drawn by some who liken the impact of service robots to that of the personal computer. Initial sales are already increasing significantly, according to IFR, which estimates that up to 2013, about 20,000 new professional service robots will be sold on average per year and that about 3 million to 3.5 million units of domestic, personal service robots will be sold.
Two key drivers pushing the rise in service robots are an aging population and a shrinking workforce, opening up opportunities both at work and at home. Generally speaking, professional service robotics is expected to serve as a workforce multiplier for increased economic growth, while domestic service robotics is expected to enable sustained personal autonomy by meeting health care needs, among others.
Despite the huge number of potential customers and users, only a few companies so far have ventured into the personal- and domestic-use segment of the robotics market. But development of such robots can be motivated by a compelling application at a competitive price, Litzenberger insists. One example is the Roomba; the latest model sells at a starting price of $300. As of 2011, iRobot sold more than 6 million home robots worldwide – including its Roomba, Scooba, Verro and Looj – and numbers are expected to double within the next four years.
“I do expect that, within the next decades, more and more service robots will be applied not to replace, but to assist, the human in its daily work,” Fischer said. “This could be especially relevant within the elderly care sector, where the robot could assist the care personnel by taking over routine tasks such as bringing the laundry to the washing machine or serving water for drinking to the elderly people.”