The big "What if"

Tracking one's steps and heart rate has never been more accessible and convenient. The advent of smartwatches, pedometers and step-counting apps has made it easy for people to track their activity, heart rate and breathing rate. With the constant evolution of technology, it’s only safe to assume that these apps and devices will also evolve in time. Currently, most people exercise with one of these devices while listening to music with another.

What if it was possible to track your heart rate and your breathing habits while listening to your favourite workout playlist? In this article, Assistant Professor Ma Dong of Computer Science at SMU has researched using Bluetooth earbuds to monitor breathing modes during running and to monitor heart rate when a person is moving. He talks about what this means for monitoring our health - and even athletic performance.

The science behind the sensing technology

Asst Prof Ma Dong's two recent pieces of research are about using wireless earbuds as wearables to detect two kinds of physiological functions. The first covers a breathing mode monitoring system when a person is running. Breathing modes refer to how a person is breathing - whether through the mouth or the nose. What this research explores is how this affects a runner’s performance, as well as how changing breathing modes affects a runner’s power and speed. 

The second physiological function he has researched is heart rate monitoring, also through using in-ear microphones. This is based on approaching every heartbeat vibration travelling through the body to the ear canal which generates a sound. Using this approach means that smart earbud devices can get more accurate heart rate readings than the average smartwatch.

“Being in motion is a very challenging scenario for a smartwatch because while a person is running, walking, or engaged in any other form of exercise, the wrist can experience a very high degree of freedom of movement. This will affect the sensor attached to the human wrist, which means you can’t always get a very accurate heart rate estimation. If you consider earbuds as a new form factor, it's relatively stable during activities because the human head cannot have such variations of movement compared to the wrist. Also, the distance from the chest to the head to the ear is closer from the heart to our wrist,” says Asst Prof Ma.

During his research, Asst Prof Ma studied a phenomenon called the occlusion effect, which is the increased sensitivity or volume of sounds produced by the body when someone’s ears are covered. Normally, the different organs in the human body create different sounds that propagate through bone conduction and escape from the opening of the ear canal. When wearing wireless earbuds, the opening of the ear canal is blocked, forming a closed space between the earbuds and the eardrum. With the canal blocked, the sound waves will be trapped within this cavity which can amplify the low-frequency sound that the device can detect.

Click to interact

“So with this phenomenon, we can explore the possibilities to enable more applications. We can do activity recognition and gesture interaction. If you can just tap somewhere on your face, you’d be able to interact with your earbuds or smartphones. We can also do speech enhancement when talking on the phone in a noisy environment. We can even authenticate who is wearing the wireless earbud. Those are just different kinds of applications we're trying to develop.” says Asst Prof Ma.

Reframing the future

When imagining a future where this technology is fully realised and publicly available, Asst Prof Ma thinks that this will be beneficial to athletic coaches. “General runners could use it to learn their limits and track their progress. While higher-level athletes could use the results in a coaching system to better achieve their targets.” For example, since the breathing modes affect a runners' performance, the runner or their coach can use the data collected by the sensors to study how to improve for maximum performance. 

There are other possible applications for this sensor technology as well. This includes user authentication, basically identifying who is wearing the earbuds. What this means is that the device can be adapted to adjust to the specific user. “Different people have different kinds of hearing capabilities, hearing preferences. You can customise the sensitivity of the sound so that it can improve and maximise their hearing experience. So that's about one application,” says Asst Prof Ma.

Yet another application of this authentication is gait analysis. “This can be for people with some kind of disease, like Parkinson's disease, who suffer from those abnormal gaits. The idea here is that when you are walking, every footstep will generate a vibration at the heel or at the forefoot. This vibration propagates through the whole body, reaching the ear canal and is amplified. The core thing here is that the amplification can lead to a better quality signal for assessment of a person’s gait.” He adds. 

This technology can also be used in partnership with other technologies to create and improve performance techniques in dance, for example. “An example could be using a combination of the microphone sensor and a motion sensor called an Inertial Measurement Unit (IMU) to measure the overall movement of ballet dancers. From this, coaches and instructors can see the timing information, the force information, and the kind of strength of the movement to create a guided coaching system. If you consider that ballet dancing is done in a group, we can have multiple sets of devices that can coordinate with each other.” Through this, an instructor can see which dancer is slower doing a certain movement, and which one should improve since the coherence of the group matters.

What's in the way?

According to Asst Prof Ma, there are two things that need to happen for this technology to be fully realised. “I think particularly for the breathing mode project, one challenge is the need to convince runners that these techniques can help them to improve their running performance. The other one is convincing developers to offer this functionality on their device.”

For future research, Asst Prof Ma hopes to collaborate with  researchers in healthcare  or exercise coaches. “I have an expertise in the use of technology for designing a guiding system for coaching or health monitoring. I would like to work with respiratory experts and exercise coaches to better understand how our design can improve a person’s running performance.