AI-Driven Imaging System Protects Photo and Video Authenticity End-to-End

To thwart sophisticated methods of altering photos and video, researchers at the New York University Tandon School of Engineering (NYU Tandon) have developed an experimental technique to authenticate images throughout the entire pipeline, from acquisition to delivery, using artificial intelligence (AI). The researchers demonstrated that neural imaging pipelines can be trained to replace the internals of digital cameras and optimized for high-fidelity photo development and reliable provenance analysis. In experiments, their prototype imaging pipeline increased the chances of detecting manipulation from approximately 45% to over 90% without sacrificing image quality.

The new approach replaces the traditional photo development pipeline with a neural network that introduces artifacts directly into the image at the moment of image acquisition. These artifacts are like digital watermarks. They are extremely sensitive to manipulation and can uncover not only the existence of photo manipulations, but also their character. The process is optimized for in-camera embedding and can survive image distortion by online photo-sharing services.

To prevent sophisticated, deep-fake methods of altering photos and video, researchers at the NYU Tandon School of Engineering devised a technique to authenticate images throughout the entire pipeline, from acquisition to delivery, using artificial intelligence (AI). In tests, a prototype pipeline increased the ability to detect manipulation from approximately 45% to over 90% without downgrading image quality. Courtesy of NYU Tandon.

“If the camera itself produces an image that is more sensitive to tampering, any adjustments will be detected with high probability,” said professor Nasir Memon. “These watermarks can survive post-processing; however, they’re quite fragile when it comes to modification. If you alter the image, the watermark breaks.”

The NYU Tandon team reasoned that modern digital imaging already relies on machine learning and that in the coming years, AI-driven processes are likely to fully replace the traditional digital imaging pipelines. As this transition takes place, Memon said, “We have the opportunity to dramatically change the capabilities of next-generation devices when it comes to image integrity and authentication. Imaging pipelines that are optimized for forensics could help restore an element of trust in areas where the line between real and fake can be difficult to draw with confidence.”

While the approach shows promise in testing, the researchers said that additional work is needed to refine the system. The solution is open-source and can be accessed at https://github.com/pkorus/neural-imaging.

The research will be presented at the Conference on Computer Vision and Pattern Recognition, June 16-20, 2019, Long Beach, Calif. (https://arxiv.org/abs/1812.01516).