MAXIMILIAN LÜCKENHAUS, MVTEC SOFTWARE GMBH
In the United States, Japan, South Korea, Germany and other European countries, the automotive industry plays a major role in the manufacturing sector. The industry is characterized by highly automated mass production, strict quality requirements and a high degree of cost sensitivity. Consequently, large manufacturers place a premium on having a close and trust-based collaborative relationship with suppliers and technology providers who support this high degree of automation in the production processes.
In that sense, industrial image processing, or “machine vision,” functions as a key technology for the automotive industry because this technology can be used to optimize a variety of processes in the value chain, such as production, quality assurance and logistics. Machine vision technologies are used in all kinds of industrial applications for the rapid and unambiguous detection of objects, resulting in the automation and acceleration of production processes. The methods are used to precisely inspect component surfaces and for identifying and automatically sorting out defective products. Similarly, the technology can also be used for completeness detection. A special feature of machine vision is its high speed — its algorithms require just milliseconds to detect and process image information. Consequently, large quantities of products can be inspected automatically and in quick succession.
Machine vision technologies optimize production processes in the automotive industry.Photo courtesy of MVTec Software.
Automotive industry as automation driver
Together with the supplier industry, the is considered an “early adopter” of machine vision technology. The industry has always promoted automation in production processes — from the production line and robot-supported manufacturing to today’s fourth industrial revolution, dubbed “Industry 4.0.” The automotive industry is also known for its considerable innovation. This can also be said of the machine vision industry, making it the ideal partner for meeting the demand for new technologies. The automotive industry is therefore the biggest customer of industrial imaging providers in Germany, accounting for 23 percent of revenue. In Europe, the total share of revenue is around 21 percent, according to a survey of VDMA (Verband Deutscher Maschinen-und Anlagenbau eV), the German engineering association.
Due to the high quality requirements, the automotive sector places a series of special demands on machine. For example, solutions must be very robust, reliable and powerful. Furthermore, enormous price pressures in the automotive and supplier industries call for competitively priced products that offer maximum value and are nevertheless suited to reduce production costs over the long term. Other vital aspects are the long-term availability of the machine vision products, including qualified support, as well as comprehensive knowledge of disparate technologies, from simple 2D measuring methods to 3D vision and image sequence analysis.
Machine vision is used for 3D surface inspection based on surface-based matching and object comparison.
One major challenge in application scenarios in the automotive industry also lies in a better integration and closer connection of machine vision components with programmable logic. By merging the two technologies more closely, car-makers can save money, speed up automated production processes and make them more efficient. Standards can also be defined and established that make it easier to integrate different components. Moreover, the increasing development of standardized interfaces increases the interoperability between machine vision and automation technology.
Machine vision — the eye of the production process
Robot-assisted processes in particular benefit from the multifaceted opportunities offered by machine vision technology because it acts as something akin to the eye of the production processes in the automotive industry. This is because a growing number of robots now feature their own cameras or, alternatively, cameras are attached to other locations and overview the complete production section from various viewpoints. Digital image information is the basis for the seamless monitoring and control of industrial production processes. At this, machine vision can be used to optimize process steps, such as determining the position of objects, securely detecting different work parts or enabling the robot to change its effectors. Moreover, different tasks can be combined, and the same cameras can be used for determining positions, planning picking processes and also quality inspection of various features, such as color or surface. The cameras are also able to read bar or data codes in combination with the machine vision software and can therefore securely identify a variety of different objects.
The use of 3D vision technologies helps increase the degree of automation in production scenarios. Until recently, the processing of 2D image data was the standard. It was used to track moving objects in the image sequence and to determine their position — but only in 2D space, such as products on a horizontal conveyor belt. The 3D coordination of objects in more complex movement processes is not possible — greatly limiting the application possibilities of the technology in the automotive production environment.
Vision technologies help to localize objects with perspective distortions.
Determining direction of movement in 3D space
MVTec addresses this need with its “3D scene flow” technology, a promising 3D vision technology also employed in the automotive industry. This solution uses a so-called multi-image camera structure, which involves the installation of two or more cameras that record image information from different angles. Every image point is given an X, Y and Z coordinate. Using a sequence of multiple images, it is now possible to move, track and exactly localize objects that are moving in three-dimensional space. The technology identifies not only the exact location of the objects but also their direction of movement and speed in space. MVTec has integrated “3D scene flow” into version 12 of the HALCON machine vision software. It is the first time that a standard software package contains such 3D-compatible algorithms for analyzing image sequences.
Almost all areas of automated car assembly benefit from the new 3D technology, including quality assurance processes, identification technologies for logistics processes, the control of robots and quality control at the final assembly stage. The technology opens up a number of entirely new possibilities for the interaction between humans and machines in automated production processes. For instance, the action radius of industrial robots is generally limited to defined production lines. If a person enters that area, sensors automatically stop the robot’s work processes, which leads to expensive disruptions in the production process.
The automotive industry is the biggest customer of industrial imaging providers in Germany, accounting for 23 percent of revenue.
Fluid and safe interaction between human and machine
3D scene flow offers the solution for this problem by using multiple cameras that monitor the robot’s work area. If someone comes near this area, the machine vision software, using multiple images, knows exactly where the person is and in which direction he or she is going. The robot only stops if the paths of human and machine are expected to cross and make a collision unavoidable. This ability to react to human trajectories reduces the number of cumbersome machine stops and production interruptions. And fewer stops mean more savings for the company. Moreover, in line with Industry 4.0 and smart factory scenarios, the automotive industry is increasingly relying on collaborative robots and autonomous forklifts that are independently moving about in production buildings. In this case, 3D-based machine vision methods reliably analyze the direction of robot and human movement, ensuring more fluid, flexible and secure interactions.
The automotive industry also benefits from 3D-based machine vision methods outside of production scenarios. The technology is also used in automatic driver assistance systems, which include sensors and cameras installed in vehicles. Using 3D image information, algorithms can reliably detect whether the car is moving properly inside its lane, whether obstacles are obstructing the road or whether other vehicles are crossing the car’s own direction of movement. These detection capabilities prevent dangerous collisions and increase traffic safety. In addition, the technology paves the way for the self-driving car.
Autonomous pallet transporters move through production halls and warehouses without collisions.
The simple way of creating machine vision applications
When it comes to the creation of machine vision applications, tools that require less programming knowledge are getting more and more important. They simplify the entire creation process and significantly increase the number of users of machine vision software. MVTec’s MERLIC development and configuration environment includes tools for the creation of complete machine vision applications, including a graphics user interface, integrated PLC communication and image acquisition based on industry standards. The central element of this software is its image-centered user interface, which intuitively guides the user through the application. Similar to a what-you-see-is-what-you-get (WYSIWYG) editor, developers can focus on the image-based display rather than on complex codes, command lines or parameter lists. The solution thus offers the ideal tool for compiling an entire machine vision application quickly and easily, including the seamless integration into automation solutions and intuitive design of the user interface.
Looking to the future
Machine vision technologies will profoundly change processes in the automotive sector. For instance, Industry 4.0 concepts will become increasingly important. In this context, machine vision methods will optimize and accelerate many aspects of industrial manufacturing. Various workpieces will be quickly and reliably detected and production processes controlled while vision technologies will improve the maintenance of production machines. Machine vision solutions will automatically detect components from mobile devices and display suitable handling information. Machine vision will accelerate and optimize logistics of international supply chains. And finally, the solutions will make the interaction between humans and machines in production processes safer.
Meet the author
Dr. Maximilian Lückenhaus is head of the Marketing + Sales business unit at MVTec Software GmbH in Munich; e-mail: [email protected].