Inauthentic artwork is a significant problem within the art world. According to the Fine Arts Expert Institute (FAEI), as many as half of the pieces in the art market are forgeries, equaling roughly $60 billion in inauthentic work. Current authentication processes, however, are often time-consuming and expensive. Forgeries are art pieces deliberately created in the style of an artist whose output is highly valued, with the intent to pass the work off as original, to deceive and perpetrate fraud. Fakes are works of art made to resemble existing ones, created by someone other than the original artist; they may or may not be illegal. Such works are sometimes very valuable in their own right. The authentication process is often long and laborious. Differing opinions cast doubt on final decisions. While the current process relies mostly on the visual skills of art experts who scrutinize pieces based on style and technique, demand for more scientific technology-based methods as part of the verification process is growing. Potential techniques include radiocarbon dating, infrared imaging, x-rays, microscopic analysis, and chemical analysis. The costs involved can be astronomical, sometimes reaching into the tens of thousands of dollars or more. The hope is that if a piece of art is authenticated, this process adds to its value by providing an objective means of certification1,2. New methods are needed to quickly authenticate artwork. Hyperspectral imaging provides a convenient and economical means to do just that, enhancing and possibly obviating the costly and sometimes destructive means currently implemented. Hyperspectral imaging Hyperspectral imaging cameras work by generating hypercubes of data, or datacubes, whereby the spectrum at each pixel in the image is collected. Subtle reflected color differences that are not observable by the human eye or even by color cameras are immediately identifiable by comparison of spectra between pixels. A variety of spectral imaging technologies currently exists. The most common type of hyperspectral imager is the pushbroom system, whereby a line on the object plane generates a 2D pattern on an array sensor. Pushbroom grating systems were the earliest forms of hyperspectral cameras, initially developed by NASA and mounted on satellites and airborne platforms for research purposes3. The collection of a complete datacube (2D spatial × 1D spectral) requires mechanical scanning. While the dispersing elements can be made small and each spectrum can be collected in as short a time as 1 ms, the mechanical motion makes these instruments somewhat bulky and prone to misalignment. Furthermore, increased spatial resolution comes at the expense of longer collection times. Band sequential, or front-staring, imagers require no mechanical scanning. In this technique, a tunable filter that can sequentially select spectral bands is placed in front of the sensor and generates the hypercube by collecting complete images at each spectral bandpass. The acquisition time does not depend on the number of pixels but rather on the number of spectral bands being acquired. These imagers are especially attractive for applications requiring high spatial and spectral resolutions with tunable spectral ranges and a small form factor. Often discussed along with hyperspectral imaging technology are multispectral systems. The most popular of these systems are based on patterned filter arrays. These arrays are an extension of color cameras in which the typical Bayer or RGB filters overlaid on the image sensor are replaced with an array of 16 or even more color filters. While no user alignment is needed, and imagers can be miniaturized, the spectral resolution is quite limited and comes at the expense of spatial resolution, making this technology inadequate for many critical sample-analysis applications. Newly developed hyperspectral cameras featuring battery-operated, hand-held staring hyperspectral cameras based on Fabry-Pérot interferometers have recently been introduced. These interferometers operate by placing two mirrors parallel to each other. By controlling the reflectivity of the mirrors and their spacing, high-finesse spectral filtering can be achieved. These types of cameras capture multimegapixel images with as many as 550 spectral bands in as few as 2 s. Moreover, the camera’s embedded hardware enables real-time processing, so users do not need to handle the large data sets typically generated by hyperspectral systems. Rather, the camera can identify features of interest in both the spectral and spatial domains. It then classifies these features in the image4. This technology can also be easily configured into form factors and configurations suitable for laboratory benchtop investigations or production line testing. Such implementation has not been possible for other band sequential techniques (for example, acousto-optic tunable filters and liquid crystal tunable filters) due to cost, reproducibility issues, environmental constraints, and power restrictions. Spectral image processing To make use of the abundance of data rendered by hyperspectral imaging, various image processing algorithms have been specially developed over the years. These are essentially mathematical techniques for deconvoluting the multiple spectral emission profiles or species, also referred to as end-members. As with the hardware, these techniques have their origins in satellite remote sensing research. The most basic and common technique is linear spectral unmixing. This method assumes spectra of each pixel is a linear combination of all end-members in the pixel and thus requires a priori knowledge such as reference spectra. Various algorithms, such as linear interpolation, are used to solve for n, number of bands, in equations for each pixel, where n is greater than the number of end-members. Another popular technique, spectral angle mapping, involves a vector representation of observed and targeted spectra to determine the closest relationships in a multidimensional space proportional to the number of bandpasses. Spectral angle mapping is widely used due to its insensitivity to brightness differences. The advent of widely accessible machine learning methods has brought a new and powerful set of tools to this endeavor. Among these are principal component analysis, a dimensionality reduction technique, and k-means clustering, a type of unsupervised learning algorithm used to find groups in the data based on any similarities in their features5. Differentiating identical works The hyperspectral datacubes of three paintings, seemingly identical except for frame and dimensions, were captured (one large original in a frame, one large unframed copy, and one small unframed copy). The subjects were placed at approximately a 1-m working distance from the hyperspectral imaging system. The imager was mounted on a standard camera tripod for stability. The objective lens presented a 19° field of view to a CMOS sensor that captured images at up to 2.3-MP spatial resolution with demosaicking. A quartz tungsten halogen lamp source and diffuser was positioned to provide uniform illumination over the area of interest. Quartz tungsten halogen sources cover a broad spectral range from the visible through the near-infrared (250 to 2500 nm) with a relatively flat spectral response, and they complement the response range of the imaging system (400 to 1000 nm). Depending on the nature and sensitivity of the pigments used in the artwork, alternate light sources such as high-brightness white LEDs that also provide broad and relatively flat spectral profiles may be preferred. These LED-based light sources are typically composite systems involving a combination of elements (e.g., RGB, Y, W) (Figure 1). Figure 1. A hyperspectral imaging camera configuration for artwork authentication (spectral reflectance). Courtesy of HinaLea Imaging. The paintings were created using different paints, but significant effort was made to make them resemble one another as much as possible in terms of reproduction style and technique. Figure 2 shows the artwork in question, with two randomly selected landmarks indicated. Figure 2. Image of artwork with two key landmarks indicated (eye and apple). Courtesy of HinaLea Imaging. To confirm that the instruments were capturing repeatable measurements of the samples, multiple measurements of the landmarks were made for each version. Scaled registration of the different size images was applied to ensure that spectra from the same relative locations were identified (Figure 3). Figure 3. Repeatability measurements of landmarks of the three versions: (from top) framed original, small copy, and large copy. Courtesy of HinaLea Imaging. A comparison of the spectra at various landmarks on the images of the three samples reveals that there are clear and distinct differences between the framed original and the copies (Figure 4). Figure 4. Comparison of spectra at eye (left) and apple (right). Courtesy of HinaLea Imaging. To efficiently assess the authenticity of a work in its entirety, classification algorithms can be applied to the datacubes. Using the spectra from the authentic piece as the reference library for the classification, a spectral angle mapping algorithm was applied to the three works. A threshold value — the spectral angle at which a pixel was considered close enough to the reference or the pixel was rejected — was adjusted and optimized by trial and error. Thus, the process compared spectra pixel by pixel, and if the spectral angle was larger than the threshold, the pixel was identified as bad. This meant the piece was labeled a counterfeit. The bad ratio was defined as the number of bad pixels divided by the total number of pixels. The bad pixels were indicated by a false color map of the artwork image (Figure 5). Figure 5. Multiple trials of classified images using spectra angle mapping to compare the authentic artwork to itself (top), and to the small (middle) and large (bottom) copies. Courtesy of HinaLea Imaging. Note that some of the areas associated with the two landmarks were flagged as bad (the eye) or good (the apple) on most of the images, even though the spectral points sample demonstrates a difference. This can be attributed to variances that straddle the tolerance of the spectral angle threshold. However, the algorithm positively and reliably distinguishes between the authentic piece and the copy in terms of the entirety of the image. The results are intuitively interpretable. This approach thus provides a means by which nonexperts can use the instrument in the process of authenticating art. By identifying distinctions in images that are not only invisible to the eye but also to color (RGB) cameras and even multispectral imagers, hyperspectral imaging cameras can provide rapid assessment of the authenticity of artwork. In the examples discussed, an advanced hyperspectral imaging system was able to quickly provide an intuitive means by which to differentiate between the original and the reproductions. This example serves to demonstrate the importance of a turnkey tool for use in art authentication. This solution can be used to augment current methods or it can potentially be used as a primary tool. The unique combination of portability and the ability to dynamically change spectral range and bandpass capability means that the same hyperspectral instrument can be configured for a wide variety of parameters and points of interest in the field. This both optimizes accuracy and reduces the time to capture the image. The semiconductor manufacturing processes involved in the fabrication of the Fabry-Pérot interferometer means that the technology is not only miniature, yielding a lightweight and compact device, but also scalable and much more economical than competing hyperspectral technologies, such as pushbroom systems, which are manually assembled in a time-consuming manner. Extension farther into the infrared range of the spectrum would enable the platform to detect and analyze hidden layers of paint, possibly unearthing even more valuable cultural treasures such as earlier iterations of the versions of a painting. Finally, the cost advantages of this technology relative to other hyperspectral imaging solutions also vastly improve access for all such potential uses, making it available to both world-renowned institutions and smaller museums. Meet the authors George Shu, Ph.D., is principal systems engineer at TruTag Technologies Inc. and HinaLea Imaging. He has a doctorate in mechanical engineering and physics from Louisiana State University; email: gshu@trutags.com. Alexandre Fong is vice president of engineering at TruTag Technologies Inc. and HinaLea Imaging. He has a B.Sc. in applied mathematics and physics, an M.Sc. in experimental physics, an MBA, and he is a chartered engineer (CEng); email: afong@trutags.com. References 1. A. Bambic (2014). Art forgery — more than half of art is fake? Widewalls, www.widewalls.ch/more-then-half-of-art-is-fake. 2. N. Larson (2014). Fine arts experts institute: lab sleuths in Geneva help art world uncover fakes, ArtDaily, www.artdaily.com/news/73562/fine-arts-experts-institute--lab-sleuths-in-geneva-help-art-world-uncover-fakes#.xvie-ohkiuk. 3. C.H. Poole (2010). Plant disease severity estimated visually, by digital photography and image analysis, and by hyperspectral imaging. Criti Rev Plant Sci, Vol. 29, Issue 2, pp. 59-107, www.doi.org/10.1080/07352681003617285. 4. H. Finkelstein and A. Fong (June 2018). Development of a hand-held high-resolution hyperspectral imaging camera. HinaLea Imaging. 5. P. Ghamisi et al. (2007). Advanced supervised spectral classifiers for hyperspectral images: a review. J Latex Class Files, Vol. 6, Issue 1.