Monitoring Heart Rate Using Smartphone Sensor Technology

Shahriar Nirjon from the University of Virginia talks to AZoSensors about monitoring the heart rate using smartphone sensor technology.

The idea of a ‘Musical Heart’ by incorporating sensor technology and music certainly is one way of bringing together wellness and entertainment. Can you describe the inspiration behind the design and development of this technology?

Whether I am driving, jogging, traveling, or relaxing, I never find the appropriate music to listen to. I believe there are many like me or maybe it’s true for everyone! The problem is that the heart wants to hear something but our music player does not understand the need. My joy was in connecting both elements together in a non-invasive and cost-effective way.   

Can you discuss how the Musical Heart works?

Musical heart is a hardware + software system. The hardware is a regular earphone with added sensors embedded inside this system such as the IR-LED, microphone and accelerometer. These sensors are able to detect the pulse (heart beat) from the ear and also the activity level (sleeping, standing, walking, jogging etc.). This information is sent to the smartphone via the audio jack interface. There is an application running on the smartphone that uses this information to suggest the right music. The more a person uses the system, the better the system adapts to the user’s reaction in response to different songs. For example, if the user wants to pump up his/her heart rate, the system will detect this current rate. Based on the user’s historical information, this system can suggest the right music to help the user in reaching that target heart rate goal.  

How does the algorithm used for this sensor technology help refine the music selection process?

We train a feed-back controller: given a target heart rate, current heart rate, and current activity level of a person, it can compute the features of music (e.g., tempo, rms energy, and pitch) that has adapted to change the user’s heart rate to a certain level. Following this, we search for a matching song in a database which could be the individual’s personal collection of songs or the cloud).

How does this system work for patients with a heart condition? Has this technology been explored in patients with a heart condition?

Not yet, but will do. The Musical Heart device has been tested on healthy people without heart condition so far.  

Is there scope for the application of this technology in medical rehabilitation?

People who will be using the Musical Heart device (e.g., for a work-out) will definitely benefit in heart rate training. However, we are exploring other forms of medical rehabilitations and hopefully that work will be published soon.  

How is the Musical Heart powered?

The earphones are powered with a thin-film battery so that the device works with any smartphone. However, with the iPhone, it is possible to harvest power directly from the phone’s audio jack and hence there is no need to use a battery to power the earphone if you have an iPhone.  

How is this sensor system customized to individual users?

Through storing historical data and learning algorithms. We simply send the entire experience of a person (statistics of heart rate – speed of the heart rate and how it varied; and statistics of activity level – what was the user’s movement or activity level) to the cloud. The cloud runs learning algorithm to update the model if necessary. For a beginner, the system uses a collaborative approach (i.e., suggests music based on similarity of the new person’s age/sex/activity/context with others in the system).

Can you discuss how this Musical Heart device compares to traditional portable heart monitors and what are the benefits?

It is convenient and non-invasive – you do not have to wear a chest wrap or use a special device to measure your heart rate while exercising. The sensors are already in the earphone, so just take your phone and keep listening music. You only specify your goal to your application (e.g., I want a calming song or I want to meet my cardio exercise program) – the rest is handled automatically. And it is cheap – the pair of ear buds we built cost $20, it should not be pricier than this, when it is commercially available.    

How do you plan on developing this research further with regard to its design and application?

I work on smartphone and embedded sensing technology. We are building a generic acoustic sensing platform for the smartphone and an array of applications, such as the Musical Heart, which will be running on the platform. So, more cool applications are coming.