CovidDeep: Testing for SARS-CoV-2/COVID-19 using Wearable Medical Sensors

Interview conducted by Joan NugentMar 21 2022

Thought LeadersNiraj K. JhaProfessor, Electrical and Computer EngineeringPrinceton University 

Can you give us an insight into your career background and your role in the research?

I received my B.Tech. from the Indian Institute of Technology, Kharagpur, India in 1981 and Ph.D. from the University of Illinois at Urbana-Champaign in 1985. I have been part of the Electrical and Computer Engineering faculty at Princeton University since 1987. My research interests span machine learning, smart healthcare, and cybersecurity. I conceived the CovidDeep concept and supervised the development of the neural network model.

Schematic diagram of the CovidDeep framework (GSR: Galvanic skin response, Ox.: oxygen saturation, BP: blood pressure, DT/RF: decision tree/random forest, NN: neural network, KB: knowledge-base, MND: multi-variate Normal distribution, GMM: Gaussian mixture model, KDE: kernel density estimation).

Can you give us an overview of the research and how CovidDeep was developed?

I conceived the idea for CovidDeep in early March 2020. CovidDeep is based on data collection using sensors (e.g., galvanic skin response, inter-beat interval, and skin temperature) embedded in a smartwatch, two discrete sensors (pulse oximetry, blood pressure), and a questionnaire (with yes/no answers for 11 questions).

The data was collected from healthy, asymptomatic, and symptomatic individuals from a hospital in Northern Italy. Physicians labeled them at that hospital to enable supervised machine learning. Then neural networks were trained using the data based on our grow-and-prune neural network synthesis procedure.

The grow-and-prune approach boosts the accuracy of the neural network significantly relative to traditional neural networks, while at the same time also significantly reducing the size of the neural network and improving its energy efficiency. The approach mimics how human brains grow from a baby to toddler to adult brain. The best neural network had 98% diagnostic accuracy for the SARS-CoV-2 virus and the resultant COVID-19 disease. A field trial with this neural network in France also yielded a high diagnostic accuracy of 97%. CovidDeep is currently awaiting FDA approval.

Image Credit: Shutterstock.com / metamorworks

Did the team encounter any issues when developing CovidDeep? If so, what were they and how did the team overcome these issues?

Data collection was done in Northern Italy when the region was suffering from a large spike in COVID-19 infections. Getting the hardware required for data collection to the Physicians was the main hurdle. The team contacted the Italian authorities, who helped facilitate this process.

An illustration of the CovidDeep processing pipeline to generate predictions from data inputs.

What are the benefits of wearable medical sensors when detecting COVID-19?

The diagnostic neural network is embedded in a smartphone app to which data from the smartwatch can be wirelessly transmitted and into which other data (from the discrete sensors and questionnaire) can be entered directly. The diagnosis can be made after just one minute of data collection. The diagnosis resides on the smartphone, thus maintaining the user's privacy. Diagnosis can be made as frequently as the user desires. Such an inexpensive, rapid real-time diagnosis could enable the economy to return to normal by quarantining only those who are infected on a given day.

The schematic diagram for pre-training of the DNN model with the synthetic dataset (DT/RF: decision tree/random forest, NN: neural network, KB: knowledge-base).

What are some of the main differences in how COVID-19 is detected via wearable medical sensors and other more traditional testing methods?

Reverse Transcription-Polymerase Chain Reaction (RT-PCR) is currently the gold standard for SARS-CoV-2 detection. This test is based on viral nucleic acid detection in sputum or nasopharyngeal swab. Although it has high specificity, it has several drawbacks. The RT-PCR test is invasive and uncomfortable, and non-reusable testing kits have led to significant supply chain deficiencies. The SARS-CoV-2 infection can also be assessed with an antibody test.

However, antibody titers are only detectable from the second week of illness onwards and persist for an uncertain length of time. The antibody test is also invasive, requiring venipuncture which, in combination with a several-day processing time, makes it less ideal for rapid mass screening. There is a great need for an alternative SARSCoV-2/COVID-19 detection method that is easily accessible to the public for repeated testing with high accuracy.

Screenshots of the CovidDeep app user experience. 

Can the research that has been carried out help diagnose any other conditions?

Yes. We have used the same approach (the collection of data through wearable medical sensors and training a neural network with the collected data) for other diseases, e.g., Type I and Type II diabetes, and mental health disorders, e.g., major depressive, bipolar, and schizoaffective disorders. We are currently investigating its applicability to several other diseases/disorders. The approach is based on the hypothesis that the disease/disorder leaves its unique signature on the physiological signals emanating from our bodies. So far, this hypothesis has been borne out.

What do the research findings mean for the potential use of wearable medical sensors in future pandemics?

It is quite likely that the same approach will also be able to diagnose infections/diseases resulting from future pandemics. It appears to be a universal approach.

Can you tell us about how NeuTigers was founded?

I introduced a course on Machine Learning for Predictive Data Analytics at Princeton University in Fall 2017. A gentleman called Adel Laoui, a resident of Princeton, attended the lectures throughout that semester. He approached me at the end of the semester to talk about research in my lab. We decided that some of the work going on in my lab, e.g., edge inference and smart healthcare, was commercializable and decided to launch NeuTigers as co-founders in the summer of 2018. Adel is CEO of the company.

What are your predictions for the use of AI in health care applications in the next 10-15 years?

This is the decade when smart healthcare (i.e., the use of machine learning in healthcare) will take off. It is already attracting VC funding that is increasing at a 40% per annum rate. Per year investments are expected to exceed $100 B in five years. Some of the major thrusts will be diagnosis (based on wearable medical sensors and images, e.g., fMRI, CT scan, X-ray, ultrasound), wellness (enabling early interventions by catching the disease, disorder or relapse from an existing condition quickly), and medical decision-making (to advise Physicians on what the optimal course of treatment should be).

Where can readers find more information?

The CovidDeep paper can be found here. Many other papers on edge analytics and smart healthcare are listed on my webpage.

About Niraj K. Jha

I am a Professor in the Department of Electrical and Computer Engineering at Princeton University. I teach courses on Machine Learning and Embedded Computing. My research group investigates algorithms and architectures for machine learning with applications in smart healthcare and cybersecurity. I am a Fellow of IEEE and ACM.

I was given the Distinguished Alumnus Award by the Indian Institute of Technology, Kharagpur, India, in 2014. I have served as the Associate Director for the Andlinger Center for Energy and the Environment at Princeton University. I have also served as the Editor-in-Chief or Associate Editor for several IEEE journals. I am a co-author of five books (including three textbooks) that are used worldwide.

I have authored or co-authored around 460 papers, among which are 14 award-winning papers. I have given several keynote speeches in the areas of smart healthcare, cybersecurity, and nanoelectronic design/test.

Figures sourced from: S. Hassantabar et al., "CovidDeep: SARS-CoV-2/COVID-19 Test Based on Wearable Medical Sensors and Efficient Neural Networks," in IEEE Transactions on Consumer Electronics, vol. 67, no. 4, pp. 244-256, Nov. 2021, doi: 10.1109/TCE.2021.3130228.