Cough-Recognition Camera Distinguishes Sounds Based on Deep Learning
The Korea Advanced Institute of Science and Technology (KAIST) and SM Instruments have developed a deep learning-based cough-recognition model that recognizes cough sounds in real time, and a camera that can track and record the sound of coughing and the location of the sound in real time. The cough-recognition camera could be used for noncontact detection of infectious diseases in public spaces or in hospitals to monitor a patient’s condition.
Diagram of how the cough-recognition model and camera work to identify, locate, and visualize coughs. Courtesy of KAIST.
The researchers applied supervised learning based on a convolutional neural network to develop the cough-recognition model. In the training and evaluation, various data sets were collected from Audioset, DEMAND, ETSI, and TIMIT. Coughing and other sounds were extracted from Audioset, and the other data sets were used as background noises for data augmentation so that this model could be generalized for various background noises in public places.
Training was conducted with various combinations of five acoustic features including a spectrogram, Mel-scaled spectrogram, and Mel-frequency cepstrum coefficients with seven optimizers. The performance of each combination was compared with the test data set.
Examples of acoustic features used to train the cough-recognition model. Courtesy of KAIST.
In tests to validate performance, the cough-recognition model achieved 87.4% accuracy. The researchers expect that the accuracy could be higher if additional learning were performed in a practical environment, such as a hospital, in the future.
The team applied the learned cough-recognition model to an acoustic camera composed of a microphone array and a camera module that collected sound. A beam-forming process was applied to the collected acoustic data to determine the direction of the incoming sound source. The integrated cough-recognition model determined whether the sound was cough. If it recognized the sound as a cough, the location of the cough was visualized as a contour image with a “cough” label at the location of the coughing sound source appearing in a video image.
Cough-recognition camera indicates coughing location in laboratory environment. Courtesy of KAIST.
A pilot test of the cough recognition camera in an office environment showed that it could distinguish cough events and other events, even in a noisy environment. In addition, it demonstrated the ability to track the location of the person who coughed and count the number of coughs in real time.
Professor Yong-Hwa Park, who led the research, said, “In a pandemic situation like we are experiencing with COVID-19, a cough-detection camera can contribute to the prevention and early detection of epidemics in public places. Especially when applied to a hospital room, the patient’s condition can be tracked 24 hours a day and support more accurate diagnoses while reducing the effort of the medical staff.”
The Center for Noise and Vibration Control at KAIST developed a coughing-detection camera that recognizes where coughing occurs and images the location. The camera can track and record information about the person who coughed, the person’s location, and the number of coughs on a real-time basis. Courtesy of KAIST.
Published: September 2020