In the coming years, embedded vision will shape the embedded market as its most important trend. One key reason, particularly in the industrial setting, will be the increasing use of compact devices such as mobile vision sensors and smart cameras that have integrated embedded software. The great advantage of these devices over permanently installed hardware is that they are easier to replace and can respond faster to different requirements. They will play an important role in the universally networked and highly automated production processes of the Industrial Internet of Things (IIoT, Industry 4.0). In such environments, numerous mobile sensors and cameras are typically installed in different locations to continuously monitor the entire production process. Machine vision contributes to universally networked and highly automated processes in the Industrial Internet of Things. Courtesy of MVTec Software GmbH. Smartphones, tablets, phablets and handheld devices also are becoming increasingly more important in the industrial setting. Some employees bring their personal smartphones to work and use them there for business purposes (referred to as bring-your-own devices, or BYOD). Not only do they use the mobile devices for on-the-job communication between colleagues, but also as compact front ends for displaying company data — effectively as an alternative to a desktop monitor. Smartphone users in production environments base their expectations on the standards they know from their personal devices. As a result, the software of industrial devices must be just as user-friendly as the software of consumer smartphones or tablets. The eyes of production Today’s mobile vision sensors and smart cameras, along with smartphones and tablets that have appropriate application software, are equipped with modern machine vision. As the “eyes of production,” these technologies perform a wide range of tasks that optimize the entire process of industrial value creation. Embedded vision software is directly integrated in mobile devices. Courtesy of MVTec Software GmbH. Machine vision technologies enable objects to be reliably detected, identified and allocated. They facilitate the automated handling of workpieces: Robots can precisely grasp, position, process, deposit and palletize these workpieces — even in three-dimensional space — thanks to 3D matching technologies. In quality control, the inspection of surfaces via machine vision allows damage, uneven areas or deformations to be reliably detected, even in large batch sizes. Rejects then can be quickly and automatically removed. Conventional desktop systems, such as a PC combined with one or multiple cameras, have served as the classic infrastructures for carrying machine vision technologies. Since machine vision systems generally perform continuous inspection and monitoring tasks, they would be permanently installed in certain locations. In the past, these architectures alone offered sufficient performance to run the powerful software functions. Today, many mobile devices are equipped with cutting-edge high-performance components, such as industrial-grade processors, high-resolution cameras and powerful memories. As a result, even complex machine vision tasks can be performed on mobile devices by using suitable embedded technologies. The inspection results of smart cameras achieved on the basis of machine vision technology also can be visualized on a conventional desktop PC. However, smart cameras require user-friendly software that allows them to be used flexibly for different applications. Requirements for algorithms There remains one difference between stationary desktop systems and mobile devices. Efficient and optimized algorithms are needed to make up for the lower computing power of mobile devices and to ensure that the machine vision software runs at high speed. Advanced embedded vision solutions, such as HALCON Embedded, are powerful enough to run robust machine vision functions on a wide variety of mobile devices. These include optical character recognition (OCR) processes, which make it possible to securely read alphanumeric codes via mobile scanners and to reliably identify objects. The OCR works even when characters are difficult to read. It is crucial that vision systems be compatible with popular and widely distributed embedded technologies. For this purpose, it must be possible to seamlessly integrate many different embedded platforms into existing environments, including the Arm processor architecture widely used in the embedded arena. A great many mobile devices — including smart cameras and vision sensors — are equipped with Arm processors. The new release 13.0.1 of the standard HALCON software makes machine vision functions available as standard for Arm-based platforms running the Linux operating system. This allows machine vision technologies to be extensively used on compact hardware devices, and it takes into account the requirements of the embedded vision growth market. IIoT Embedded vision systems can be used profitably in the IIoT context. One important application is predictive maintenance. Consider the following scenario: A large number of smart sensors with integrated machine vision software is installed in the surroundings of large machine fleets. They continuously monitor production and immediately report any abnormalities apparent in a system. These can range from an indication of a pending malfunction to a total failure of a machine. To prevent expensive production shutdowns, the service team can react specifically to the warning signals at an early stage. Embedded vision technologies optimize the maintenance of machines and systems. Courtesy of MVTec Software GmbH. Another option is to use thermal imaging cameras complete with machine vision software. Analysis of the recorded image data in real time can determine if a machine is overheating and if significant damage to the machine is imminent. Technicians then can use this knowledge to take proactive measures and avoid a worse situation. Embedded vision technology in mobile devices also can be used to help with the maintenance and upkeep of machines and systems. Of particular use are smartphones, phablets or tablets based on the Android operating system. If a component in a control cabinet is defective and needs to be replaced, for example, a service technician can simply point the camera of his mobile phone toward the rack and take a snapshot. The machine vision software integrated into the smartphone detects the corresponding component and provides the generated digital data for further maintenance. The software also can support the creation of a multimedia manual. By reliably detecting a wide range of components with a mobile device, operating instructions and other additional information can be conveniently downloaded online. This reduces the rate of errors and optimizes the maintenance process. Because of the increasingly widespread use of mobile devices in the networked production environment of the Industrial Internet of Things, embedded vision technologies are becoming more important. These technologies provide the entire range of functions of a robust machine vision software solution, particularly on mobile vision sensors and smart cameras. Embedded vision thus makes it possible to evaluate digital image data without problems on compact, powerful and energy-efficient computing platforms. In this way, machine vision is penetrating applications that have not previously been accessible to smart cameras or PC-based systems. The value creation is shifting from hardware to software. As a result, standard software solutions with industrial applications are becoming more important. After all, they enable users to benefit from features they are already familiar with from PC-based systems. All in all, high-performance embedded vision technologies contribute to efficient and highly automated processes along the entire industrial value chain of the IIoT. Meet the author Johannes Hiltner is product manager for HALCON at MVTec Software GmbH; email: info@mvtec.com.