AI Analyzes Images Taken Under Severe Conditions, Increases Clarity

A multimodal image fusion technology uses artificial intelligence (AI) to automatically combine visible images taken by standard cameras with nonvisible images taken by specialized devices such as thermal or terahertz cameras.

The technology uses AI to conduct detailed examinations of each image in order to assess the degree of visibility of each of its parts. It then automatically extracts the best areas from each image, taking into consideration environmental characteristics such as brightness, the direction of light, and obstacles in the field of vision.

The technology further uses AI to analyze clues that could enable it to detect an abnormality or hazard in a non-visible camera image. It automatically generates a multimodal image with exceptionally high visibility, while regulating the degree of enhancement to avoid causing image breakdowns, such as clipped highlights and crushed blacks.

In order to analyze nonvisible and visible images of the same subject using conventional technology, images need to be viewed and compared separately. Abnormalities or hazards contained in nonvisible images can be more easily overlooked when images are combined manually.

NEC Corporation and Tokyo Institute of Technology are jointly developing a multimodal image fusion technology that dramatically improves the clarity of images by using artificial intelligence. Courtesy of NEC Corporation.

Courtesy of NEC Corporation.

“Until now, a specialist had to manually carry out complex conversion tasks in order to combine images taken by different types of cameras. This technology eliminates the need for such manual work, using AI to effectively and automatically combine images taken by different cameras. This also increases visibility by actively utilizing the strong points of each visible image and nonvisible image, even when the images are difficult to visualize,” said professor Masatoshi Okutomi, Tokyo Institute of Technology.

“As this technology enables instant visual clarification, even under harsh conditions, it allows users to make well informed evaluations. For example, it can be applied to monitoring systems to assist with nighttime observations, or to infrastructure inspection devices to improve the detection of interior and exterior abnormalities, such as cracking,” said Akio Yamada, general manager, Data Science Research Laboratories, NEC Corporation.

NEC and Tokyo Institute of Technology (Tokyo Tech) presented the technology at the 23rd Symposium on Sensing via Image Information, held on June 7-9, 2017 at Pacifico Yokohama, Nishi-ku, Yokohama City, Japan.