Device Uses AI, Thermal Imaging to Survey Public Areas for Flu-Like Illness

A contactless, portable surveillance device, developed by a team at the University of Massachusetts Amherst (UMass), can capture crowd-level bioclinical signals directly related to physical symptoms of influenza-like illnesses from public waiting areas in an unobtrusive manner in real time, then analyze the data to directly monitor flu-like illnesses and influenza trends. The health surveillance device, called FluSense, is powered by artificial intelligence (AI). It could be used in hospitals, waiting rooms, and larger public spaces to forecast seasonal flu and other viral respiratory outbreaks, such as the COVID-19 pandemic or SARS.

FluSense uses a microphone array and a thermal camera along with a neural computing engine to passively and continuously characterize speech and coughing sounds along with changes in crowd density on the edge. It stores no personally identifiable information, such as speech data or distinguishing images.

The FluSense device houses these components. Courtesy of UMass Amherst.

To build the device, the researchers first developed a lab-based cough model. They then trained a deep neural network classifier to draw bounding boxes on thermal images representing people and to count them. “Our main goal was to build predictive models at the population level, not the individual level,” professor Tauhidur Rahman said.

The researchers placed FluSense, which is about size of a large dictionary, in four health care waiting rooms at the UMass University Health Services clinic. From December 2018 to July 2019, the FluSense platform collected and analyzed more than 350,000 thermal images and 21 million nonspeech audio samples from the public waiting areas.

FluSense was able to accurately predict daily patient counts with a Pearson correlation coefficient of 0.95. The researchers compared signals from FluSense with laboratory-confirmed influenza case data obtained in the same facility and found that the FluSense sensor-based features were strongly correlated with laboratory-confirmed influenza trends.

The early symptom-related information captured by FluSense could support current influenza prediction efforts, such as the FluSight Network, a multidisciplinary consortium of flu forecasting teams. “I thought if we could capture coughing or sneezing sounds from public spaces where a lot of people naturally congregate, we could utilize this information as a new source of data for predicting epidemiologic trends,” Rahman said.

Tauhidur Rahman (left) and Forsad Al Hossain display their FluSense device. Courtesy of UMass Amherst.

Researcher Forsad Al Hossain said that FluSense, which consists of an edge-computing sensor system, models, and data processing pipelines, demonstrates the value of combining AI with edge computing to enable data to be gathered and analyzed right at the data’s source. “We are trying to bring machine-learning systems to the edge,” he said. “All of the processing happens right here [in the FluSense device]. These systems are becoming cheaper and more powerful.”

The next step is to test FluSense in other public areas and geographic locations. “We have the initial validation that the coughing indeed has a correlation with influenza-related illness,” professor Andrew Lover said. “Now we want to validate it beyond this specific hospital setting and show that we can generalize across locations.”

The research was published in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Tech (www.doi.org/10.1145/3381014).