ENABLING HUMAN PHYSIOLOGICAL SENSING BY LEVERAGING INTELLIGENT HEAD-WORN WEARABLE SYSTEMS

Abstract

This paper explores the challenges of enabling human physiological sensing by leveraging head-worn wearable computer systems. In particular, we want to answer a fundamental question, i.e., could we leverage head-worn wearables to enable accurate and socially acceptable solutions to improve human healthcare and prevent life-threatening conditions in our daily lives? To that end, we will study the techniques that utilise the unique advantages of wearable computers to (1) facilitate new sensing capabilities to capture various biosignals from the brain (EEG), the eyes (EOG), facial muscles (EMG), sweat glands (EMG), and blood vessels (EMG), (2) address motion artefacts and environmental noise in real-time with signal processing algorithms and hardware design techniques, and (3) enable long-term, high-fidelity biosignal monitoring with efficient on-chip intelligence and pattern-driven compressive sensing algorithms. Based on these developments, the paper will discuss potential applications that can be enabled, such as microsleep detection, epileptic seizure monitoring, and blood pressure measurement.

Keywords: human biosensing, mobile healthcare, on-chip machine learning, wearable đo đạc sinh học từ cơ thể người, chăm sóc sức khoẻ di động, học máy trên chip, thiết bị đeo được

1. Introduction

The wearable healthcare market is experiencing significant growth in recent years, reaching over $121 billion globally in 2021 and is predicted to surpass $390 billions by 2030 (Zoting, 2024). With more than 500 million units shipped annually, head-worn wearables such as headphones, earphones, headbands, hats, etc., hold an enormous potential to be the platforms of new innovations (Falkous & Callaway, 2018). Furthermore, they are already socially accepted to be a part of our daily life. As a result, introducing more sensing modalities into these wearables is less likely to obstruct the user’s everyday activities, resulting in easy adoption for newly developed research and technology in this field (Choudhury, 2021).

The human head is a prime location for wearable computers on the human body. It houses essential organs and tissues such as the brain, the eyes, facial muscles, blood vessels, and sweat glands. By listening to their generated biosignals and interactions, we can unlock numerous novel healthcare applications. For example, using just facial muscle signals (i.e., electromyography (EMG)) alone, one can infer the stress level of a user (Tsai, Chou, Gale, & McCall, 2002; Lundberg et al., 1994) and the eating habit and the type of food consumed by the user (Kohyama, Mioche, & Bourdio3, 2003; Mioche, Bourdiol, Martin, & No¨el, 1999; Kohyama, Mioche, & MARTIN, 2002). When combining these EMG signals with brain signals (i.e., electroencephalogram (EEG)), one can further understand the user’s emotional states (X.-W. Wang, Nie, & Lu, 2014; Mu¨ller et al., 2008), their pain and suffering level (Sarnthein, Morel, Von Stein, & Jeanmonod, 2003), or sedation level during surgery (De Deyne et al., 1998). Even the current gold standard for sleep quality monitoring in hospitals is based on brain signals (EEG), chin muscle signals (EMG), and eye movement signals (i.e., electrooculography (EOG)) (Kushida et al., 2005; Bloch, 1997) captured from the head. The signals can also be used for measuring attention deficit hyper-activity disorder (ADHD) (Murias, Swanson, & Srinivasan, 2007; Jonkman et al., 1999), or detecting autism onset (Bernier et al., 2007).

Figure 1: a) A standard biosignals monitoring system and an w earable

system (Emotiv Inc.)

However, most existing sensing methods are cumbersome, intrusive, and expensive, primarily suitable only for stationary and short-term usages such as in clinics or hospitals. For example, existing solutions, such as EEG sensor caps (“actiCHamp”, 2024; “BIOPAC Electrode Cap”, 2024), provide very high resolution and accuracy but require users to carry a bulky data collection hub and to wear tens to hundreds of electrodes on their heads (Fig. 1a). Other existing wearable devices such as Emotiv (Fig. 1b) (“Emotiv brainwear”, 2024), NeuroSky MindWave (“NeuroSky MindWave”, 2024), BrainLink Pro (“BrainLink Pro”, 2024), Muse (“Muse”, 2024), Kokoon (“Kokoon”, 2024a), Neuroon Open (“Neuroon Open”, 2024a), Naptime (“Neuroon Open”, 2024b), etc., which were designed to capture a small number of these signals, are required to be worn on the user’s face or head. They are inconvenient, intrusive, and less likely to be adopted as a long-term and everyday wearable solution despite the potential benefits they can bring.

This paper explores the challenges of enabling human physiological sensing by leveraging head-worn wearable computers. In particular, we want to answer a fundamental question, i.e., could we leverage head-worn wearables to enable accurate and socially acceptable solutions to improve human healthcare and prevent life-threatening conditions in our daily lives? To that end, we will study the techniques that utilise the unique advantages of wearable computers to (1) facilitate new sensing capabilities to capture various biosignals from the brain, the eyes, facial muscles, sweat glands, and blood vessels, (2) address motion artefacts and environmental noise in real-time at the electrical level, and (3) enable long-term, high-fidelity biosignal monitoring with efficient on-chip intelligence and pattern-driven compressive sensing algorithms. While proposed sensing techniques focus on specific applications such as blood pressure monitoring, microsleep detection, and epileptic seizure monitoring, the developed software and hardware platforms could be used for a wide range of applications in the mobile healthcare and human-computer interaction domains where long-term and reliable physiological and cognitive sensing is essential. The potential examples include sleep improvement, focus monitoring, worker safety management, eating habit monitoring, closed-loop pain coaching, etc.

2. Literature Review

2.1. Behind-the-ear biosignal sensing and microsleep detection

Microsleep detection application. More than 65 million people in the U.K. and the U.S. suffer from Excessive Daytime Sleepiness (EDS) due to sleep deprivation, obstructive sleep apnea, and narcolepsy (Pagel, 2009; Hafner, Stepanek, Taylor, et al., 2017). EDS often results in frequent lapses in awareness of the environment (i.e. microsleeps). Healthy people with sleep deprivation usually experience microsleep (Pagel, 2009). Shift workers, night-time security guards, and navy sailors with sleep problems have a 1.6x higher risk of being injured, causing 13% of all work injuries (Uehli, Mehta, et al., 2014). Sleepy drivers are at a 3x higher risk of an accident, causing one in five fatal car crashes (Dawson, Reynolds, et al., 2018). People with sleep apnea also suffer from microsleep. The microsleep issue due to sleep apnea alone leads to a loss of nearly $150 million every year due to daily work performance reduction and vehicle accidents (of Sleep Medicine, n.d.). Additionally, more than half of Narcoleptic people are unemployed because of uncontrollable microsleep (Jennum, Knudsen, et al., 2009). They often use Amphetamines to keep themselves awake, resulting in many drug overdose cases (Centers, 2018). Combined, the sleepiness problem of drivers and the workforce costs the U.K. and U.S. economies more than $411 billion annually (Europe, n.d.; Hafner et al., 2017). As a result, it is an urgent need for an unobtrusive, reliable, and socially acceptable microsleep detection solution throughout the day.

Understanding microsleep. The Orexin system is a wakefulness network throughout the whole central nervous system, as illustrated in Fig. 2 (left). It promotes neuron activity in the mid-brain, the cerebrum, and the visual cortex. These neuronal activities are represented through brain waves, such as fast Beta (β) and Alpha (α) waves while the brain is wakeful and conscious and the slow Theta (θ) waves when the brain experiences sleepiness. Furthermore, studies on animals (Takahashi, Lin, & Sakai, 2008) have shown that Orexin neurons modulate pupil size, eyelid position, and possibly convergence and eye alignment via motoneurons of multiple muscle fibres. As a result, the wakefulness state is also represented by the movements and activities of the eyes. Additionally, several studies (Sakurai, 2007) have shown that Orexin regulates wakefulness in the autonomic nervous system (ANS) by activating the ANS through projections to the ventrolateral medulla (VLM) and spinal cord, causing the inhibition of sleep. The changes in sympathetic tone are, in turn, represented by changes in facial muscles and sweat gland activity.

Figure 2: Wakefulness neural pathway (left) and standard electrode locations (right).

Microsleep is the temporary episode of losing consciousness and is the key to capturing the transition from wakefulness to sleep. A microsleep episode can last from a few to 30 seconds, and people can still wake up after an episode. Microsleep manifests itself both behaviorally (slow-rolling eyes, gradual eye-lid closure, head nods (Peiris et al., 2006)) and electrically (shift in electroencephalography (EEG) from fast α and β waves to slower θ activities (Paul et al., 2005)). These manifestations link to the inhibition of the Orexin system. Microsleep is extremely dangerous for tasks requiring constant awareness since people who experience MS are usually unaware of them and still believe that they are awake the whole time (Higgins & Fette, 2012). This often happens with people who have excessive daytime sleepiness (EDS).

Conventionally, the need for placing multiple sensors on the user’s head to capture different biomarkers for accurate microsleep detection makes it challenging to build a wearable and socially acceptable system. As illustrated in Fig. 2 (right), several electrodes (e.g. at least 9 in the standard polysomnography (PSG) system (Center, n.d.)) are usually needed to be placed on the user’s scalp to capture brain waves. A wearable camera or 2-4 biopotential electrodes can be placed on the user’s eyes to capture eye movements. To capture facial muscle contractions, electrodes are placed on the user’s chin. Lastly, sweat gland activity is often captured by electrodes on the wrist or the fingers. With this number of sensors at different locations on the user’s head and face, achieving wearability and social acceptability for microsleep detection is not a trivial task. These studies confirm that there are four key biomarkers that we need to capture for microsleep detection. The remaining questions are (1) where to place the sensors, (2) how many sensors are sufficient, and (3) how the sensors can be made to capture this information? We will discuss these questions in Sec. 3.1.

Motion artefacts. Motion artefact is a significant roadblock that prevents practical usage of biosignal sensing systems. EEG and EOG signals are heavily affected by motion due to their low amplitude (i.e., microvolts). There are two main sources of motion, i.e., (1) micromovements of the sensing electrodes and (2) triboelectric effects on the measurement cables. The electrode-skin contact and the connection between sensing electrodes to the readout circuit could be modelled as a series of capacitors and resistors (Chi, Jung, & Cauwenberghs, 2010). During motion, fluctuations in this electrical pathway result in unwanted voltage potentials that superimpose with the biosignals. After analogue-to-digital conversion, motion artefacts are not distinguishable from meaningful signals. Thus, various methods are needed on multiple components of the sensing systems, including electrodes, readout circuits, and signal processing, to be effective (Seok, Lee, Kim, Cho, & Kim, 2021). Software methods such as Independent Components Analysis (ICA) and Blind Source Separation (BSS) are often employed to mitigate motion artefacts. However, the requirements for real-time processing and a minimal number of sensors make it difficult to employ these methods on wearable devices (Soldati, Calhoun, Bruzzone, & Jovicich, 2013).

Environmental noise. Noise coupling from the environment is another critical issue for biosignal sensing systems. Environmental noise could come from various sources, such as main power lines, power transformers, motors, etc. Almost all electrical equipment connected to the main power line has a power transformer. These transformers convert high-voltage alternative current (120/240 VAC) to low-voltage current. During operation, the alternative magnetic field generated by transformers could easily interfere with biosignal sensing circuits, creating unpredictable noise. Similarly, electric motors also generate strong magnetic fields that could induce tiny noise currents inside a sensing circuit (Macy, 2015). The main power line is another significant source of environmental noise. Depending on the country, these power lines are typically 120/240 VAC at 50/60Hz. This AC couples into the measurement through capacitive coupling, where the human body acts as a coupling medium (Daniel & Neagu, 2018). Since the main power line is not perfectly sinusoidal, we often observe strong power of harmonics, i.e., up to 5th-order from the fundamental frequency (Macy, 2015). Since environmental noise could be multiple orders of magnitude stronger than biosignals, differential measurement is employed to eliminate the common noise and extract small differential biosignals. Common-mode rejection ratio (CMRR) is the metric to quantify the ability to suppress common noise of a measurement system (Teplan et al., 2002).

2.2 Long-term head-based biosignal monitoring on low-power wearables

The importance of long-term biosignal monitoring. The wearable healthcare market has been experiencing significant growth in recent years, and it is predicted that healthcare wearable devices will be the next generation of personal telemedicine practice. This is especially important for patients with chronic diseases and after surgery, where constant monitoring is essential to prevent fatalities (Lou, Wang, Jiang, Wei, & Shen, 2020). However, many wearable-enabled healthcare applications have not been deployed due to limited battery lifetime, slow response rate, and inadequate biosignal quality.

The trade-off between signal fidelity, response time, and battery life is a long-standing challenge for wearable devices (Casson, 2019; Ometov et al., 2021). In many healthcare applications, the wearable usually takes the role of data collecting device due to their limited energy and computing resources (Patel, Park, Bonato, Chan, & Rodgers, 2012). The collected data are transmitted to nearby mobile devices through wireless communications (e.g., Bluetooth, WiFi) to predict emergency events or upload to users’ healthcare providers for further diagnosis. Though maintaining the collected signal fidelity is crucial (Hu et al., 2015), continuous wireless communication has a high cost on the battery life (Gurve, Delisle-Rodriguez, Bastos-Filho, & Krishnan, 2020). E.g., Bluetooth could consume up to several mWs (Groups, n.d.), while WiFi could go as high as 10s of mW (Moreno-Cruz et al., 2020), depending on the data rate. As a result, many healthcare wearables must reduce signal quality (i.e., by lowering data rate) and increase response latency (i.e., by increasing communication intervals) to improve battery lifetime (Gurve et al., 2020).

Event Sparsity. We observe that the events of interest (e.g., seizures, microsleeps, etc.) are important but rarely happen. Several studies have reported that these events only occur less than 5% of the signal duration (S. J. Smith, 2005). Thus, detecting these events on the device could help to cut a significant amount of energy needed to stream the signals out. However, detecting these events requires multiple signal modalities (i.e., EEG, EOG, EMG, etc.) and a complex algorithm, making it challenging to implement on resource-constrained devices. Our intuition is that we could decompose these complex events into smaller and generic patterns of interest (PoIs). For example, an epileptic seizure waveform could consist of EEG spike/polyspike and slow-wave (focal/generalised non-specific seizures), 3-Hz spike-and-wave discharges (absence seizures) and stiffing and convulsion patterns (tonic-clonic seizures). Similarly, we can decompose a microsleep event into alpha, theta wave, slow eye movements, and muscle contraction patterns on the EEG, EOG, and EMG signals. Thus, it is feasible to detect these patterns directly on the device with an efficient pattern recognition technique.

Signal Sparsity. We also observe that the sparsity property also presents at the signal level. While biosignals are known to be non-sparse in time or frequency domains, they could have sparse representations in other domains (e.g., wavelets). Thus, we do not need all the collected samples to reconstruct the signal. The compressive sensing (CS) theory has been developed to exploit signal sparsity. It states that the number of signal measurements depends on inherent information contained in the signal and is much lower than the Nyquist rate (Gurve et al., 2020). The effectiveness of CS relies directly on finding a reliable domain with high sparsity. However, this is still an open challenge for non-stationary biosignals (Abdulghani, Casson, & Rodriguez-Villegas, 2012).

From these observations, we hypothesise that by exploiting both event and signal sparsity, the amount of data reduction could be significant, leading to a highly energy-efficient system. However, we must take great care in designing such a system. With the constrained computing resources of wearable devices, any additional energy spent on complicated algorithms could easily outweigh any benefits from the reduced wireless transmission. The remaining questions are (1) How can we develop the pattern detection models so that they can be both accurate and efficient? (2) How can we devise a compressive sensing method that could achieve both low sampling rate while maintaining high signal fidelity? and (3) How can we optimise the system to ensure the efficiency of additional computation? We will discuss these questions in Sec. 3.2.

2.3 In-ear blood vessels activity and pressure monitoring

Frequent blood pressure measurement. Blood pressure (BP) is one of the foremost vital signs measured when patients first arrive in the hospital, as BP can provide doctors with insight to initiate their diagnosis. For example, chronic kidney disease, sleep apnea, and adrenal and thyroid disorders can all cause high BP, while low BP indicates the possibility of heart or endocrine problems, dehydration, severe infection, or even blood loss. Additionally, uncontrolled elevated BP is a major symptom of many life-threatening diseases, such as hypertension, heart failure or stroke (“HBP and the Cardiovascular System”, n.d.). Until recently, the reliable way to measure BP was done by a health care practitioner (HCP) such as a physician, doctor, or nurse. The clinician wraps an arm cuff around the patient’s upper arm and rapidly inflates the cuff with air. Once the cuff has reached maximum inflation, the systolic and diastolic pressures are determined by slowly releasing the air from the cuff and observing the pulse sound with a stethoscope over the brachial artery below the cuff. Since the invention of digital BP devices, non-medical trained users can self-measure their BP at home, as an acoustic sensor can replace the stethoscope, and a pressure sensor with a DC pump can substitute the pressure gauge and hand pump.

However, these devices often cause discomfort and inconvenience for those who need frequent BP monitoring, such as haemodialysis patients (Miskulin & Weiner, 2017), individuals with undiagnosed white coat hypertension or undiagnosed masked hypertension (O’Brien, 2003), which have a prevalence of 15-30% (Franklin, Thijs, Hansen, O’Brien, & Staessen, 2013) and 16.8% (Ogedegbe, Agyemang, & Ravenell, 2010) in the US, respectively. There is also increased use of frequent BP monitoring for post-operative organ transplant recipients (Ramesh Prasad, 2012), with more than 30,000 solid organ transplants occurring every year (D. of Health & Services, 2018). In such cases, BP is measured every 30 minutes for 24 hours (Drawz, Abdalla, & Rahman, 2012), while each hemodialysis session takes around four hours. Therefore, there is a significant need for an unobtrusive and comfortable BP monitoring approach.

Fundamental of BP measurement. Starting with a brief overview of existing blood pressure (BP) monitoring widely used today, we will point out current limitations, providing context for our novel approach. Measuring blood flow pressure can be done with both invasive and non-invasive methods. Although invasive approaches deliver highly accurate results, it is costly and only available in hospitals. Non-invasive techniques are far more useable, as their process is quick, low cost, and relatively simple. However, these non-invasive techniques are set to cycle or must be done manually, unlike continuous measurements provided by invasive arterial methods.

Artery deformation under the effects of cuff pressure grants the key signatures to estimate BP, as shown in Fig. 2.3. When the cuff pressure is equal to the systolic pressure (SBP), blood flow continues through the occluded artery, but only the highest arterial pressure can be detected. On the other hand, if the cuff pressure is lower than the diastolic pressure (DBP), the detected pulse is very weak. Using auscultatory methods, medical practitioners listen to pulse sound propagation through a stethoscope to determine BP. Oscillatory, on the other hand, was developed for the digital device by estimating BP from the change of pulse amplitude. It detects the Maximum Pulse Amplitude (MPA) AM first and applies predefined fractions of the peak amplitude ratio AM /AS and AM /AD to detect where the systolic and diastolic pressure occurs and uses these values to infer the pressure. AS and AD are the amplitude of systole and diastole, respectively. Unlike auscultatory methods, oscillatory methods do not need to completely occlude the blood vessel in order to detect the systolic BP (Geddes, Voelz, Combs, Reiner, & Babbs, 1982), which is well-suited for our balloon model. However, current oscillation ratios only apply to the arm or wrist BP measurement model. Therefore, they are not eligible for our in-ear case. Generating a new in-ear ratio requires a large-scale data set, including an invasive method to measure BP from inside the ear, which is infeasible. Instead, we propose a technique to measure BP without applying the characteristic ratios. To achieve this goal, we thoroughly examine the change of amplitude with respect to the change of cuff pressure. Then, we extract the key properties and formulate them into mathematical equations for processing. According to (“Guest Commentary: How blood-pressure devices work”, n.d.), during the deflation:

• Pulse amplitude increases when the cuff pressure is close to the systolic level. The increment increases more quickly when the pressure reaches and passes through the systolic point.

• At the systolic and diastolic cycle cross-section, the amplitude obtains its highest value (the MPA).

• Amplitude rapidly decreases once the pressure passes the MPA and moderately decreases once it reaches the diastole point. The DBP position occurs at the highest decreasing amplitude.

Figure 3: The response of blood artery to outer pressure that causes the measurement of BP.

These observations provide key insights for composing the solutions to detect MPA, SBP, and DBP. In particular, the diastolic position is the minimum of the downslope amplitude, and MPA is the peak of the amplitude, as shown in Fig. 3. We can derive the systolic location as the maximum upslope amplitude. However, sometimes our in-ear balloon pressure might not reach the systolic phase due to comfort requirements. Therefore, we have to rely on the relational equation between MPA, SBP, and DBP (Baker, Westenskow, & Ku¨ck, 1997):

PM = βPS + (1 − β)PD,

where β is the systole ratio of the cardiac cycle and PM , PS and PD are the MPA, SBP, and DBP respectively. Most literature reports β as a fixed value (Mafi, Rajan, Bolic, Groza, & Dajani, 2012; S. Lee, Jeon, & Lee, 2013) and is widely accepted, but each person can have a slightly different ratio dependent on age, gender, and health condition. Moreover, incorrect estimation of β increases the estimation error, as noticed in (Moran et al., 1995). In our eBP system, we propose an adaptive estimation for β based on the pulse-wave form. Realising the importance of frequent BP monitoring and the discomfort of current devices, we design eBP as ear-worn equipment to (1) capture the pulse signal inside the ear from a balloon-attached pulse sensor and (2) use this information to estimate the BP.

3. Results and discussions

3.1. Sensing head-based biopotentials and microsleep detection with a behind-the-ear wearable

This section presents WAKE, a novel behind-the-ear wearable device for microsleep detection. By monitoring biosignals from the brain, eye movements, facial muscle contractions, and sweat gland activities from behind the user's ears, WAKE can detect microsleep with a high temporal resolution. We introduce a Three-fold Cascaded Amplifying (3CA) technique to tame the motion artefacts and environmental noise for capturing high-fidelity signals. Through our prototyping, we show that WAKE can suppress motion and environmental noise in real-time by 9.74-19.47 dB while walking, driving, or staying in different environments, ensuring that the biosignals are captured reliably. We evaluated WAKE using gold-standard devices on 19 sleep-deprived and narcoleptic subjects. The Leave-One-Subject-Out Cross-Validation results show the feasibility of WAKE in microsleep detection on an unseen subject with average precision and recall of 76% and 85%, respectively.

Challenges. To realise WAKE, we face the following key challenges: (1) heavy noise created by motion and coupled from the environment in daily use is the long-standing challenge limiting the practical uses of wearable biosignal sensing systems, as it is difficult to ensure high fidelity signals; (2) making a wearable, and socially-acceptable device that can capture microsleep is non-trivial because multiple sensors are usually needed to capture its core biomarkers; (3) microsleep detection from behind the ears is an unexplored topic where existing techniques cannot be applied directly; and (4) the BTE biosignals are weak and overlap with each other in the three-orders magnitude range.

Figure 4: Key contributions of the behind-the-ear biosensing wearable system.

Contributions. The key findings and contributions of this work are presented in Fig. 4.

• We devise a Three-fold Cascaded Amplifying (3CA) hardware technique to make it more practical by ensuring high-fidelity signals while mitigating motion and environmental noise.

• We identify and localize the minimum number of areas behind human ears where biomarkers from the brain, the eyes, facial muscles, and sweat glands can be captured reliably for microsleep detection.

• We design and prototype a wearable, compact, and socially acceptable device that can capture multiple head-based physiological signals. A user study on the participants shows that more than 85% of people praises its unobtrusiveness and is willing to wear it during their daily lives.

• Using a wide range of microsleep biomarkers as features, we developed a hybrid model of a hierarchical classification model and EMG-event-based heuristic rule to detect users' microsleep.

• We evaluate the proposed system using our custom-built prototype on 19 subjects. 10 of them are healthy, 14 of them have mild to severe Excessive Daytime Sleepiness and 1 of them is diagnosed with the narcoleptic problem.

• In Leave-One-Subject-Out Cross Validation (LOSOCV), the system obtains 76% precision and 85% recall, showing the feasibility of microsleep detection of WAKE on an unseen subject.

3.2. Efficient Pattern-dRiven cOmpressive Sensing for Low-Power Biosignal-based Wearables

In this section, we explore the challenges of building a new event-driven compressive sensing framework, called PROS, that could enable energy-efficient wearables for biopotential-based applications. We develop PROS based on the sparsity nature of biosignals and events. Specifically, PROS consists of tiny pattern recognition primitives and a pattern-driven compressive sensing algorithm that work together to significantly reduce the transmission rate while maintaining high-fidelity signals. PROS also enables the ability to react to critical events immediately on the device. While we currently focus on EEG, EOG, and EMG biosignals and a head-worn form factor, PROS is also applicable for a variety of healthcare wearable devices such as smartwatches, earphones, smart clothes, etc., where achieving continuous, high-fidelity biosignal streaming, low-latency responses, and long battery life is critical to their applications. 

Challenges. To realise PROS, we face the following challenges: (1) biosignal events (e.g., seizures, microsleep, pain, etc.) require multimodal sensing channels and a complex algorithm (e.g., machine learning) to detect, which is not feasible on low computing resource wearable; (2) we lack a reliable domain with high sparsity to compress biosignals on the device effectively; (3) low power wearable devices have extremely constrained computing resource, i.e., an MHz microcontroller (MCU) and KBs of system memory, making it challenging to deploy advanced computations without consuming significant energy.

Figure 5: Key results of the on-chip pattern recognition and pattern-driven compressive sensing.

Contributions. To overcome the challenges, we make the following contributions (Fig. 5): 

• We identify the pattern primitives of biosignals such as EEG, EOG, and EMG and develop tiny recognition models (TinyPR) for continuous on-chip detection and low-latency responses.

• We devise a pattern-driven compressive sensing (PDCS) technique to efficiently compress the captured signal pattern with appropriate wavelet domains, boosting the compression factor and recovered signal quality.

• We design a hardware platform and employ optimisation techniques in both hardware and OS levels to support advanced signal processing and neural network operations of PROS.

• The prototype of PROS is evaluated on two open datasets of PROS subjects. In a practical use case such as epileptic seizure detection, PROS can reduce the data rate by 24X, boost the power efficiency by more than 1200%, and enable real-time responses within 10s of milliseconds while maintaining high-fidelity signals.

3.3. eBP-Sensing in-ear blood vessels to monitor blood pressure with a wearable computer

This section presents a device called eBP to measure BP from inside the user's ear, aiming to minimise the measurement's impact on users' normal activities while maximising its comfort level. eBP has three key components: (1) a light-based pulse sensor attached to an inflatable pipe that goes inside the ear, (2) a digital air pump with a fine controller, and (3) a BP estimation algorithm. In contrast to existing devices, eBP introduces a novel technique that eliminates the need to block the blood flow inside the ear, which alleviates the user's discomfort. We prototyped eBP custom hardware and software and evaluated the system through a comparative study on 35 subjects. The study shows that eBP obtains the average error of 1.8 mmHg and -3.1 mmHg and a standard deviation error of 7.2 mmHg and 7.9 mmHg for systolic and diastolic, respectively. These errors are around the acceptable margins regulated by the FDA's AAMI protocol, which allows average errors of up to 5 mmHg and a standard deviation of up to 8 mmHg.

Challenges. Realising eBP has the following challenges. (1) Existing non-invasive algorithms cannot directly be applied to in-ear BP measurements. In-ear BP monitoring is an unexplored topic in which many of the existing techniques cannot be applied. Even the feasibility of the technique has yet to be confirmed. (2) The mechanism enabling the use of an inflatable balloon to measure BP from inside the ear is non-trivial. When the balloon inflates, the sensor should attach firmly to the ear canal and not slide out. In addition, applying insufficient pressure will result in an inaccurate BP measurement, while applying too much pressure may cause discomfort or hurt the ear canal. (3) Blocking the artery is an important process in measuring BP, yet it is challenging to do that inside the ear. Applying a weak pressure will result in an inaccurate BP measurement, and applying a strong pressure may discomfort the user or hurt the ear canal. (4) The in-ear pulse signals are weak and buried under noise. In addition, the motion artefacts are difficult to remove and can impact BP measurement accuracy. (5) BP measurements are sensitive to the contact quality between the sensor and in-ear skin; yet maintaining consistent contact pressure is difficult.

Figure 6: Key results of the proposed in-ear blood pressure measurement system.

Contributions. In this work, we make the following contributions (as illustrated in Fig. 6). First, we propose a novel concept of in-ear frequent BP monitoring and show that it is not only feasible but also comfortable. Second, we propose a blocking-free optical-oscillometric approach to allow the in-ear sensor to measure important parameters in BP measurements (i.e., systolic amplitude and diastolic amplitude). Third, we prototype a device with a custom-built circuit and hardware/software components for in-ear BP measurements. In particular, we customise an off-the-shelf catheter to safely insert it into the ear canal with a light pulse sensor attached. In addition, we build a portable in-ear wearable device to control the light pulse sensor and the catheter to capture the pulse signal (i.e., BP) accurately and reliably. Fourth, we devise an algorithm to process and qualify the highly noisy pulse signals captured from inside the ear to ensure high-quality BP measurements. Lastly, we conducted a study with 35 users and verified the performance of the proposed system using an FDA-approved BP measurement device (KonQuest KBP-2704A).

4. Conclusion

In this paper, we explore the challenges of enabling human physiological sensing by leveraging head-worn wearable computers to improve human healthcare and prevent life-threatening conditions that can be used in our daily lives. In particular, we present our studies on techniques that utilise the unique advantages of wearable computers to facilitate new sensing capabilities to capture various biosignals from the brain, the eyes, facial muscles, sweat glands, and blood vessels. We also propose techniques to address motion artefacts and environmental noise in real-time at the electrical level. Furthermore, we develop a framework to enable long-term and high-fidelity biosignal monitoring with efficient on-chip intelligence and a novel pattern-driven compressive sensing algorithm. Finally, we have demonstrated the usability of the proposed methods in three practical use cases such as microsleep detection, blood pressure measurement, and epileptic seizure monitoring.

Acknowledgement

I would like to thank my advisors (Prof Tam Vu, Prof Niki Trigoni, Prof Andrew Markham) and all my collaborators (Prof Cecilia Mascolo, Prof Farnoush Banaei-Kashani, Prof Thang Dinh, Dr VP Nguyen, Dr Nam Bui, Dr Hoang Truong, Dr Minh Tran, Dr Anh Nguyen, Dr Hong Jia, Dr Dong Ma, Dr Ann Halbower, Dr Robin Deterding, Dr Tuan Dinh, Dr Zohreh Raghebi, Dr Young Kwon, Dr Taeho Kim, Carole Kline, and Tuan Nguyen), who have contributed to the projects. The work described in this thesis was partially funded by the University of Oxford DPhil Scholarship, the Alfred P. Sloan Fellowship no. FG-2020-13110 (TV), ERC through Project 833296 (EAR), Nokia Bell Labs, NSF#2132112, NSF CNS/CSR#1846541, NSF SCH#1602428, Google Faculty Awards 2018, the Schramm Foundation and the Colorado Advanced Industries Accelerator (AIA).

References

1. Abdulghani, A. M., Casson, A. J., & Rodriguez-Villegas, E. (2012). Compressive sensing scalp eeg signals: implementations and practical performance. Medical & biological engineering & computing , 50 (11), 1137–1145.

2. actichamp [Computer software manual]. (2024). (https://goo.gl/uagGNX)

3. Baker, P. D., Westenskow, D. R., & Ku¨ck, K. (1997, May 01). Theoretical analysis of non-invasive oscillometric maximum amplitude algorithm for estimating mean blood pressure. Medical and Biological Engineering and Computing , 35 (3), 271–278.

4. Bernier, R., Dawson, G., Webb, S., & Murias, M. (2007). Eeg mu rhythm and imitation impairments in individuals with autism spectrum disorder. Brain and cognition, 64 (3), 228–237.

5. Biopac electrode cap [Computer software manual]. (2024). (https://goo.gl/GGBrJv)

6. Bloch, K. E. (1997). Polysomnography: a systematic review. Technology and health care, 5 (4), 285–305.

7. Boucsein, W. (2012). Electrodermal activity. Springer Science & Business Media. Brainlink pro [Computer software manual]. (2024). (https://goo.gl/hrxuAb)

8. Casson, A. J. (2019). Wearable eeg and beyond. Biomedical engineering letters, 9 (1), 53–71.

9. Center, T. P. R. (n.d.). Procedure manual for polysomnography [Computer software manual]. https://tinyurl.com/yc9ptdjz.

10. Centers, A. A. (2018). Narcolepsy and how substance abuse effects it [Computer software manual]. (https://tinyurl.com/yxh4hh7z)

11. Chi, Y. M., Jung, T.-P., & Cauwenberghs, G. (2010). Dry-contact and noncontact biopotential electrodes: Methodological review. IEEE reviews in biomedical engineering, 3 , 106–119.

12. Choudhury, R. R. (2021). Earable computing: A new area to think about. In Proceedings of the 22nd international workshop on mobile computing systems and applications (pp. 147–153).

13. Daniel, T¸ . D., & Neagu, M. (2018). Cancelling harmonic power line interference in biopotentials. Compendium of New Techniques in Harmonic Analysis; IntechOpen: London, UK , 19.

14. Dawson, D., Reynolds, A. C., et al. (2018). Determining the likelihood that fatigue was present in a road accident: a theoretical review and suggested accident taxonomy. Sleep medicine reviews.

15. De Deyne, C., Struys, M., Decruyenaere, J., Creupelandt, J., Hoste, E., & Colardyn,

16. F. (1998). Use of continuous bispectral eeg monitoring to assess depth of sedation in icu patients. Intensive care medicine, 24 (12), 1294–1298.

17. Dong, X., Ye, W., Tang, Y., Wang, J., Zhong, L., Xiong, J., . . . Feng, Z. (2021). Wakefulness-promoting effects of lateral hypothalamic area–deep brain stimulation in traumatic brain injury-induced comatose rats: Upregulation of α1adrenoceptor subtypes and downregulation of gamma-aminobutyric acid β receptor expression via the orexins pathway. World Neurosurgery , 152 , e321– e331.

18. Drawz, P. E., Abdalla, M., & Rahman, M. (2012, April). Blood pressure measurement: clinic, home, ambulatory, and beyond. American journal of kidney diseases : the official journal of the National Kidney, 60 (3), 449-462.

19. Emotiv brainwear [Computer software manual]. (2024). (https://goo.gl/uagGNX)

20. Europe, R. (n.d.). Lack of sleep costing uk economy up to £40 billion a year [Computer software manual]. (https://tinyurl.com/3x9bpfau)

21. Falkous, C., & Callaway, J. (2018). Wearable technology in life insurance [Computer software manual]. (https://tinyurl.com/3ypnb3de)

22. Franklin, S., Thijs, L., Hansen, T., O’Brien, E., & Staessen, J. (2013, December). White-coat hypertension: New insights from recent studies. Hypertension, 62 (6), 982-987.

23. Geddes, L. A., Voelz, M., Combs, C., Reiner, D., & Babbs, C. F. (1982, Nov 01). Characterization of the oscillometric method for measuring indirect blood pressure. Annals of Biomedical Engineering , 10 (6), 271–280.

24. Go, S. A., Coleman-Wood, K., & Kaufman, K. R. (2014). Frequency analysis of lower extremity electromyography signals for the quantitative diagnosis of dystonia. Journal of electromyography and kinesiology , 24 (1), 31–36.

25. Groups, B. S. W. (n.d.). Bluetooth core specification 4.0 [Computer software manual]. (https://tinyurl.com/2e25vsxu)

26. Guest commentary: How blood-pressure devices work [Computer software manual]. (n.d.). https://tinyurl.com/y5rmewuf.

27. Gurve, D., Delisle-Rodriguez, D., Bastos-Filho, T., & Krishnan, S. (2020). Trends in compressive sensing for eeg signal processing applications. Sensors, 20 (13), 3703.

28. Hafner, M., Stepanek, M., Taylor, J., et al. (2017). Why sleep matters—the economic costs of insufficient sleep: a cross-country comparative analysis. Rand health quarterly , 6 (4).

29. Hbp and the cardiovascular system [Computer software manual]. (n.d.).

30. https://tinyurl.com/ybv4vlst.

31. Higgins, L., & Fette, B. (2012). Drowsy driving.

32. Hu, B., Peng, H., Zhao, Q., Hu, B., Majoe, D., Zheng, F., & Moore, P. (2015). Signal quality assessment model for wearable eeg sensor on prediction of mental stress. IEEE transactions on nanobioscience, 14 (5), 553–561.

33. Jennum, P., Knudsen, S., et al. (2009). The economic consequences of narcolepsy.

34. Journal of Clinical Sleep Medicine, 5 (03), 240–245.

35. Jonkman, L., Kemner, C., Verbaten, M., Van Engeland, H., Kenemans, J., Camfferman, G., . . . Koelega, H. (1999). Perceptual and response interference in children with attention-deficit hyperactivity disorder, and the effects of methylphenidate. Psychophysiology , 36 (4), 419–429.

36. Kohyama, K., Mioche, L., & Bourdio3, P. (2003). Influence of age and dental status on chewing behaviour studied by emg recordings during consumption of various food samples. Gerodontology , 20 (1), 15–23.

37. Kohyama, K., Mioche, L., & MARTIN, J.-F. (2002). Chewing patterns of various texture foods studied by electromyography in young and elderly populations. Journal of Texture Studies, 33 (4), 269–283.

38. Kokoon [Computer software manual]. (2024a). (https://goo.gl/tr3QJV) 

39. Kushida, C. A., Littner, M. R., Morgenthaler, T., Alessi, C. A., Bailey, D., Coleman Jr, J., . . . others (2005). Practice parameters for the indications for polysomnography and related procedures: an update for 2005. Sleep, 28 (4), 499–523.

40. Lee, S., Jeon, G., & Lee, G. (2013, 10). On using maximum a posteriori probability based on a bayesian model for oscillometric blood pressure estimation. Sensors (Basel, Switzerland), 13 , 13609-23.

41. Li, Y., Poon, C. C., & Zhang, Y.-T. (2010). Analog integrated circuits design for processing physiological signals. IEEE Reviews in Biomedical Engineering , 3 , 93–105.

42. Lou, Z., Wang, L., Jiang, K., Wei, Z., & Shen, G. (2020). Reviews of wearable healthcare systems: Materials, devices and system integration. Materials Science and Engineering: R: Reports, 140 , 100523.

43. Lundberg, U., Kadefors, R., Melin, B., Palmerud, G., Hassm´en, P., Engstro¨m, M., & Dohns, I. E. (1994). Psychophysiological stress and emg activity of the trapezius muscle. International journal of behavioral medicine, 1 (4), 354–370.

44. Macy, A. (2015). The handbook of human physiological recording. Referenced , 7 , 2017.

45. Mafi, M., Rajan, S., Bolic, M., Groza, V. Z., & Dajani, H. R. (2012, Aug). Blood pressure estimation using maximum slope of oscillometric pulses. In 2012 annual international conference of the ieee engineering in medicine and biology society (p. 3239-3242).

46. Malhotra, R. K., & Avidan, A. Y. (2013). Sleep stages and scoring technique. Atlas of sleep medicine, 77–99.

47. Mioche, L., Bourdiol, P., Martin, J.-F., & No¨el, Y. (1999). Variations in human masseter and temporalis muscle activity related to food texture during free and side-imposed mastication. Archives of Oral Biology , 44 (12), 1005–1012.

48. Miskulin, D. C., & Weiner, D. E. (2017). Blood pressure management in hemodialysis patients: What we know and what questions remain. Seminars in Dialysis, 30 (3), 203-212.

49. Moran, D., Epstein, Y., Keren, G., Laor, A., Sherez, J., & Shapiro, Y. (1995, 12). Calculation of mean arterial pressure during exercise as a function of heart rate. Applied human science : journal of physiological anthropology , 14 , 293-295.

50. Moreno-Cruz, F., Toral-Lo´pez, V., Escobar-Molero, A., Ru´ız, V. U., Rivadeneyra, A., & Morales, D. P. (2020). trench: ultra-low power wireless communication protocol for iot and energy harvesting. Sensors, 20 (21), 6156.

51. Mowrer, O. H., Ruch, T. C., & Miller, N. E. (1935). The corneo-retinal potential difference as the basis of the galvanometric method of recording eye movements. American Journal of Physiology-Legacy Content , 114 (2), 423–428.

52. Mu¨ller, K.-R., Tangermann, M., Dornhege, G., Krauledat, M., Curio, G., & Blankertz, B. (2008). Machine learning for real-time single-trial eeg-analysis: from brain–computer interfacing to mental state monitoring. Journal of neuroscience methods, 167 (1), 82–90.

53. Murias, M., Swanson, J. M., & Srinivasan, R. (2007). Functional connectivity of frontal cortex in healthy and adhd children reflected in eeg coherence. Cerebral Cortex , 17 (8), 1788–1799.

54. Muse [Computer software manual]. (2024). (https://goo.gl/5zwtcJ)

55. Nayak, C. S., & Anilkumar, A. C. (2020). Eeg normal waveforms. statpearls [Computer software manual]. Treasure Island, FL: StatPearls Publishing. http://www. ncbi. nlm. nih. gov . . . .

56. Neuroon open [Computer software manual]. (2024a). (https://goo.gl/5WyTuj) Neuroon open [Computer software manual]. (2024b). (https://goo.gl/iY3Ber) Neurosky mindwave [Computer software manual]. (2024).

57. (https://goo.gl/cEf7fi)

58. O’Brien, E. (2003). Ambulatory blood pressure monitoring in the management of hypertension. Heart , 89 (5), 571–576.

59. of Health, D., & Services, H. (2018). Organ donation and transplantation [Computer software manual]. https://data.hrsa.gov/topics/health-systems/organ-donation.

60. of Sleep Medicine, A. A. (n.d.). Economic burden of undiagnosed sleep apnea in u.s. is nearly $150b per year [Computer software manual]. (https://tinyurl.com/yxkfe5zh)

61. Ogedegbe, G., Agyemang, C., & Ravenell, J. E. (2010, October). Masked hypertension: evidence of the need to treat. Current hypertension reports, 12 (5), 349-355.

62. Ometov, A., Shubina, V., Klus, L., Skibin´ska, J., Saafi, S., Pascacio, P., . . . others (2021). A survey on wearable technology: History, state-of-the-art and current challenges. Computer Networks, 193 , 108074.

63. Pagel, J. (2009, May). Excessive daytime sleepiness [Computer software manual]. (https://www.aafp.org/afp/2009/0301/p391.html)

64. Patel, S., Park, H., Bonato, P., Chan, L., & Rodgers, M. (2012). A review of wearable sensors and systems with application in rehabilitation. Journal of neuroengineering and rehabilitation, 9 (1), 1–17.

65. Paul, A., et al. (2005). Variability of driving performance during microsleeps. Peiris, M. T., et al. (2006). Frequent lapses of responsiveness during an extended visuomotor tracking task in non-sleep-deprived subjects. Journal of sleep research, 15 (3), 291–300.

66. Ramesh Prasad, G. V. (2012, December). Ambulatory blood pressure monitoring in solid organ transplantation. Clinical Transplantation, 26 , 185-191.

67. Sakurai, T. (2007). The neural circuit of orexin (hypocretin): maintaining sleep and wakefulness. Nature Reviews Neuroscience, 8 (3), 171.

68. Sarnthein, J., Morel, A., Von Stein, A., & Jeanmonod, D. (2003). Thalamic theta field potentials and eeg: high thalamocortical coherence in patients with neurogenic pain, epilepsy and movement disorders. Thalamus & Related Systems, 2 (3), 231–238.

69. Seok, D., Lee, S., Kim, M., Cho, J., & Kim, C. (2021). Motion artifact removal techniques for wearable eeg and ppg sensor systems. Frontiers in Electronics, 2 , 685513.

70. Smith, C. J., & Havenith, G. (2011). Body mapping of sweating patterns in male athletes in mild exercise-induced hyperthermia. European journal of applied physiology , 111 (7), 1391–1404.

71. Smith, S. J. (2005). Eeg in the diagnosis, classification, and management of patients with epilepsy. Journal of Neurology, Neurosurgery & Psychiatry, 76 (suppl 2), ii2–ii7.

72. Soldati, N., Calhoun, V. D., Bruzzone, L., & Jovicich, J. (2013). Ica analysis of fmri with real-time constraints: an evaluation of fast detection performance as function of algorithms, parameters and a priori conditions. Frontiers in human neuroscience, 7 , 19.

73. Svensson, U. (2004). Blink behaviour based drowsiness detection: method development and validation. Statens v¨ag-och transportforskningsinstitut.

74. Takahashi, K., Lin, J.-S., & Sakai, K. (2008). Neuronal activity of orexin and nonorexin waking-active neurons during wake–sleep states in the mouse. Neuroscience, 153 (3), 860–870.

75. Teplan, M., et al. (2002). Fundamentals of eeg measurement. Measurement science review , 2 (2), 1–11.

76. Thakor, N. V. (2017). Biopotentials and electrophysiology measurement. Measurement, Instrumentation, and Sensors Handbook , 64–1.

77. Tsai, C.-M., Chou, S.-L., Gale, E. N., & McCall, W. D. (2002). Human masticatory muscle activity and jaw position under experimental stress. Journal of oral rehabilitation, 29 (1), 44–51.

78. Uehli, K., Mehta, A. J., et al. (2014). Sleep problems and work injuries: a systematic review and meta-analysis. Sleep medicine reviews, 18 (1), 61–73.

79. Wang, X.-W., Nie, D., & Lu, B.-L. (2014). Emotional state classification from eeg data using machine learning approach. Neurocomputing , 129 , 94–106.

80. Yoruk, A., Bisognano, J. D., & Gassler, J. P. (2016). Baroreceptor stimulation for resistant hypertension. American journal of hypertension, 29 (12), 1319–1324.

81. Zangiabadi, N., Ladino, L. D., Sina, F., Orozco-Hern´andez, J. P., Carter, A., & T´ellezZenteno, J. F. (2019). Deep brain stimulation and drug-resistant epilepsy: a review of the literature. Frontiers in neurology , 601.

Zoting, P. (2024). Wearable technology market global industry analysis, size, share, growth, trends, regional outlook, and forecast 2022-2030 [Computer software manual]. (https://tinyurl.com/ja3kdaa4)