An upsurge of portable, consumer-facing devices at the intersection of smartphone computing and spectroscopy is now leveraging integration.
FAROOQ AHMED, SCIENCE WRITER
The seminal 1960s television series “Star Trek” delivered futuristic inventions and Captain James T. Kirk’s roguish smile into American homes on a weekly basis. Among the show’s memorable gadgets were the communicator, which inspired the design of the clamshell cellphone, and the tricorder — a portable, Swiss army knife-like gadget that allowed the crew of the USS Enterprise to diagnose medical conditions in seconds with the flash of a handheld probe across a patient’s body. Despite decades of attempts, no one device has completely replicated the original tricorder’s capabilities.
Portable analysis
Progress for consumer-oriented devices is being made. According to Brian Cunningham, professor of electrical and computer engineering at the University of Illinois at Urbana-Champaign, “the ‘Star Trek’ tricorder was absolutely a touchstone as we developed our device — the TRI Analyzer.” His group is one of several across the world exploiting the convergence of two increasingly prevalent technologies — smartphone computing and spectroscopy — for portable analysis.
The LinkSquare device is an easy-to-use, handheld spectrometer that connects to a smartphone via Wi-Fi. Here, the device analyzes the cocoa content of chocolate. Courtesy of Stratio Inc.
The TRI Analyzer is a 3D-printed cradle that physically attaches to a smartphone and uses the phone’s internal camera as a visible-light spectrometer (Figure 1). “Cellphones today are very well-engineered,” Cunningham said. “And their hardware and software are very well-integrated.”
This point-of-care device detects the photon emission spectrum of liquids that are used in three common biological diagnostic tests: an enzyme-linked immunosorbent assay for fetal fibronectin protein, which is indicative of preterm birth; the phenylketonuria screening for newborns; and an assay for the soluble transferrin receptor, which is a biomarker of iron deficiency. The results were comparable to those from clinical-grade spectrometers1,2.
Figure 1. The TRI Analyzer, inspired by the “Star Trek” tricorder, detects color changes in biological assays. Courtesy of Kenneth Long.
“We view it as a portable lab,” Cunningham said. “In practice, we could adapt the analyzer to thousands of already developed tests. It would have a lot of benefit where [patients] may not have access to them.”
His research group took an intrinsic approach to creating the TRI Analyzer, leveraging the smartphone’s existing components rather than modifying the phone or using an external spectrometer. An optical fiber steers light from the phone’s LED flash into a liquid compartment in a microfluidic chamber. The light shines through a liquid sample or off an antibody-linked photonic crystal, bounces off a mirror on the opposite side of the compartment, and then travels through a second optical fiber onto a diffraction gradient that spreads the light spectrum across the internal CMOS active-pixel sensor. An app on the phone analyzes the incoming video to detect the subtle color changes that occur in diagnostic tests.
The cradle itself is low-cost and should have widespread appeal.
“This approach is just better for the biological assays we have in mind, which generate colored products in the visible light range,” he said. “We anticipate making these cradles for popular phone models.”
Earlier this year, the Austin, Texas-based medical diagnostic company Reliant Immune Diagnostics licensed the TRI Analyzer and plans on integrating it into its services.
Democratizing discovery
Fuhong Cai of Hainan University in Haikou, China, concurs with Cunningham’s assessment on the state of portable spectroscopic technology. He expects that eventually “the imaging spectrometer will replace the ordinary camera in many applications.”
However, the price of current spectrometers is prohibitively high. To address cost and “democratize” access to the technology, Cai, together with Sailing He’s group from Zhejiang University in Hangzhou, China, developed a handheld, smartphone-connected spectrometer for less than $300 using off-the-shelf components (Figure 2a).
Figure 2. Fuhong Cai’s handheld spectrometer connects to a Wi-Fi camera and displays spectral images on a smartphone (a). Cai’s compact imaging spectrometer attaches to an unmanned aerial vehicle (b). Courtesy of Fuhong Cai and Gaoao Ye, Wuhan Hongxiang Information Technology Co., China.
Instead of relying on the internal smartphone camera, Cai and colleagues opted for an external approach to marrying these technologies. Their pen-like device consists of optical lenses and a diffraction grating, all housed within a metal tube the length of a smartphone. It weighs 140 g, which is about the same as a baseball. The device has a resolution of approximately 17 nm in a range of 400 to 675 nm.
The research group used the handheld spectrometer to image biological samples, including chlorophyll in bananas and myoglobin in pork. They also created — in about 16 s — a 3D spectral image data
cube of hemoglobin in the human hand3. They are deploying the device in biomedical imaging and environmental testing and are installing the spectrometer on unmanned aerial and underwater vehicles (Figure 2b).
Cai said this current work is proof of concept. As the cost of IR cameras decreases, he and his colleagues hope to extend their range. Moving to the IR will “provide more meaningful optical information, which is in accordance with the concept of ‘big data’ in this [scientific] era.”
Managing data
Data was one worry that the USS Enterprise’s doctor, “Bones” McCoy, didn’t seem to have when he pulled out the tricorder and waved it over a wounded officer. But Cai and others, such as the California- and South Korea-based company Stratio Inc., realize that the information both acquired and generated by the devices is key to long-term viability.
Similar to Cai’s spectrometer, Stratio’s LinkSquare is an external pen-like device that connects to a smartphone app via Wi-Fi. According to Su Ryeo Oh, chief strategy officer at Stratio, the device
operates in the visible and NIR, up to 1000 nm, with an optical resolution of about 5 nm. It’s a sleek instrument that arrives in an iPhone-like case and feels like an oversized fountain pen in one’s hand. The LinkSquare app contains applets that allow users to verify the authenticity of medications4, discern rotten from fresh tuna, identify certain types of skin cancer, and even estimate the percentage of cocoa in various chocolates. The app currently has about 15 applets — a number that is increasing. The device was born from the Stratio founders’ research at Stanford University into IR imaging (Figure 3).
Figure 3. The LinkSquare device (a) is an easy-to-use, handheld spectrometer that connects to a smartphone via Wi-Fi. Above, the device analyzes the freshness of meat (b), and the authenticity of medicine (c). Courtesy of Stratio Inc.
Oh said data integrity is ensured by designing both the hardware and software in-house. “We run machine learning algorithms ourselves, and we always make sure that the applets are above 95 percent accuracy,” she said. “We do not use crowdsourced data.”
Both Oh and Cai agree that artificial intelligence (AI) is crucial to leveraging information in the big data era of science. Cai said that He’s group at Zhejiang University is planning to use AI to “make spectral image data smarter,” while Stratio is using their machine learning algorithms to engineer applets for clients if they upload their data to the company’s servers. Stratio has also produced a software development kit that allows developers to build their own apps. Oh reveals that in September 2018, they will release an NIR LinkSquare (800 to 1200 nm) and eventually a SWIR-capable device (800 to 1600 nm) with a germanium detector instead of the more traditional indium gallium arsenide (InGaAs) detector. The additional capabilities will produce even more interesting data, Oh said.
Integrating the spectrometer
Although the processing power of today’s smartphones rival that of past supercomputers, the integrated cameras have received incremental upgrades.
“The performance of the mobile phone’s camera is gradually improving, but it is still not suitable for imaging spectrometers due to its short focal length and small sensor area,” Cai said. It’s one reason why he and companies such as Stratio have developed external smartphone spectrometers.
University of Illinois professor Cunningham takes a more agnostic approach, noting that he doesn’t “have a religious belief about either [internal or external spectrometers], because in some projects, we’re taking the extrinsic approach.”
Technology, however, waits for no one, and a new spectrometer chip from Andrea Fiore’s lab at Eindhoven University of Technology in the Netherlands points at a solution to fully integrate spectrometers into smartphones — one that could allow the industry to take a step closer to the elusive tricorder, or at least a communicator on steroids.
Fiore, a professor of applied physics and leader of the university’s Photonics and Semiconductor Nanophysics group, squeezed a spectrometer onto a 10- × 10-µm chip — much tinier than a grain of sand. “It’s essentially a very small microelectromechanical system, integrated with a filter and a photodetector,” he said (Figure 4a).
Figure 4. An electron microscope reveals the fabricated microspectrometer, with detail of the light-capturing cavity (a). Sketch image of the microspectrometer concept. The photonic crystal sits beneath the upper membrane (blue perforated plane) (b). Courtesy of Andrea Fiore.
The sensor contains two membranes, spaced 200 nm apart, and an optical cavity that filters a small frequency range of the incoming light. The detector, a PIN photodiode, is similar to those commonly used in communications and sensing. By applying a voltage, the top membrane can be electromechanically displaced, which allows for a tunable and narrow wavelength detection — up to 30 nm in the NIR5. Fiore’s group used the spectrometer to measure an absorption line of hydrofluoric acid gas, which has a linewidth of only 16 pm. “Typically, you need a very large spectrometer to measure this type of line,” Fiore said. “But because of our high resolution, we could see it” (Figure 4b).
He is in talks with smartphone manufacturers to learn more about what they want but admits that the sensor will need to detect a broader spectrum before it will be commercially viable. Still, at 10 ×
10 µm, dozens of Fiore’s chips would fit in a phone.
“Things are moving very fast, and probably there will be products on the market based on simpler technologies coming up quite soon,” he said. Fabrication cost is not a factor. “Essentially the [sensor] isn’t more complicated than semiconductor lasers, which you can buy for a couple of dollars or even less. Of course, you will need high volumes to get to that price range.”
Fiore is certain that for consumer technologies, external devices separate from smartphones will be a hard sell, because “practically no one wants to carry a second device.”
Cunningham, whose TRI Analyzer was named after the “Star Trek” device, sees a robust future for smartphone spectroscopy. “The iPhone 10 has four cameras — two back-facing, one selfie camera, and the face ID camera. In the future, why couldn’t there be a spectrometer that’s devoted toward medical diagnostics, the color of your couch, [and] an entire range of undreamed applications?”
Acknowledgments
The author would like to thank Su Oh, Stratio Inc.; Fuhong Cai, Hainan University, Haikou, China; Andrea Fiore, Eindhoven University of Technology, Netherlands; and Brian T. Cunningham, University of Illinois at Urbana-Champaign.
References
1. K.D. Long et al. (2017). Multimode smartphone biosensing: the transmission, reflection, and intensity spectral (TRI)-analyzer. Lab Chip, Vol. 17, Issue 19, pp. 3246-3257.
2. R.D. Peterson (2017). Comparison of methods study between a photonic crystal biosensor and certified ELISA to measure biomarkers of iron deficiency in chronic kidney disease patients. Sensors, Vol. 17, Issue 10, p. 2203, doi:10.3390/s17102203.
3. F. Cai et al. (2017). Pencil-like imaging spectrometer for bio-samples sensing. Biomed Opt Express, Vol. 8, Issue 12, pp. 5427-5436, doi:10.1364/BOE.8.005427.
4. K.J. Baik et al. (2017). Pharmaceutical tablet classification using a portable spectrometer with combinations of visible and near-infrared spectra. Ninth International Conference on Ubiquitous and Future Networks (ICUFN), pp. 1011-1014, doi:10.1109/ICUFN.2017.7993951.
5. Ž. Zobenica et al. (2017). Integrated nano-opto-electro-mechanical sensor for spectrometry and nanometrology. Nat Commun, Vol. 8, Issue 1, article no. 2216, doi:10.1038/s41467-017-02392-5.