Medical Engineering and Physics 37 (2015) 499–504
Contents lists available at ScienceDirect
Medical Engineering and Physics journal homepage: www.elsevier.com/locate/medengphy
Technical note
Unobtrusive monitoring and identification of fall accidents Pepijn van de Ven a,∗, Hugh O’Brien a, John Nelson a, Amanda Clifford b a b
Department of Electronic and Computer Engineering, University of Limerick, Limerick, Ireland Department of Clinical Therapies, University of Limerick, Limerick, Ireland
a r t i c l e
i n f o
Article history: Received 3 October 2013 Revised 18 December 2014 Accepted 16 February 2015
Keywords: Fall sensing Falls prevention Accelerometry Ambient assisted living Mobile health
a b s t r a c t Falls are a societal and economic problem of great concern with large parts of the population, in particular older citizens, at significant risk and the result of a fall often being grave. It has long been established that it is of importance to provide help to a faller soon after the event to prevent complications and this can be achieved with a fall monitor. Yet, the practical use of currently available fall monitoring solutions is limited due to accuracy, usability, cost, and, not in the least, the stigmatising effect of many solutions. This paper proposes a fall sensor concept that can be embedded in the user’s footwear and discusses algorithms, software and hardware developed. Sensor performance is illustrated using results of a series of functional tests. These show that the developed sensor can be used for the accurate measurement of various mobility and gait parameters and that falls are detected accurately. © 2015 IPEM. Published by Elsevier Ltd. All rights reserved.
1. Introduction In this paper a sensor proof of concept is presented that offers a solution to a range of issues encountered in traditional fall sensors. A third of older people over the age of 65 years fall in the community each year with prevalence rates increasing to over a half of individuals over 80 years [1]. Falls related injuries come at a high cost to society and individuals. Society as a whole is faced with high annual bills in the order of €400 million for a country such as Ireland and this is predicted to increase to €1 billion by 2020 [2]. Following a fall, individuals may be impeded in their daily lives by the fear of falling, reduced self-efficacy and the physical consequences of the fall [3]. A high proportion of uninjured fallers (47%) are unable to get up from the floor after a fall [4] due to lack of strength and physical fitness [5]. An inability to get up from the floor after a fall can result in a long lie, which is defined as remaining on the floor for more than an hour after a fall. The long lie is associated with co-morbidities including dehydration, pneumonia, hypothermia and pressure sores and has been found to significantly increase the probability of death (50%) within 6 months [3,6]. Hence, detection of a fall is of great importance and numerous fall detection systems have been developed. Most wearable fall detection systems rely on detecting impact. This, however, is not always a feature of significance in a fall. Moreover, device discreteness and obtrusiveness negatively impact on the acceptability and usability of many current devices [7].
∗
Corresponding author. Tel.: +353 61202925. E-mail address: [email protected] (P. van de Ven).
http://dx.doi.org/10.1016/j.medengphy.2015.02.009 1350-4533/© 2015 IPEM. Published by Elsevier Ltd. All rights reserved.
This paper presents a sensor concept that offers a solution to these issues. The presented sensor is a footwear based sensor that uses pressure, acceleration and temporal information to assess the user’s ambulatory parameters and the incidence of falls. The use of footwear based sensors has been investigated by various groups [8,9].1,2 However, the main focus of these efforts has been on falls prevention through monitoring of gait and mobility parameters. Although falls risk assessment is an important aspect of the management of falls, it will never be possible to fully rule out the occurrence of a fall. Hence, the detection of the fall event itself will remain of crucial importance in any falls emergency system. The focus on falls prevention for footwear based sensors is likely for more than one reason. As the old proverb says “prevention is better than cure”, but a likely further reason for the focus on falls prediction for footwear based sensors is that in this location a fall event is difficult to identify with the classical fall detection approach of measuring acceleration and impact. In this work we propose methods to obtain accurate fall detection from the user’s foot by using a state machine which allows the measured pressure and acceleration to be assessed within the context of the current user physical activity. This strategy allows for a relatively low sampling frequency, which in turn results in lower battery consumption. The sensor would be relatively cheap to manufacture and as the concept allows for a solution fully embedded in the user’s footwear, is fully unobtrusive. Moreover, unlike 1 WIISEL – Wireless Insole for Independent and Safe Elderly Living (http://www.wiisel.eu/). 2 Balance Problems? Step into the iShoe (http://web.mit.edu/newsoffice/2008/ i-shoe-0716.html).
500
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
Fig. 1. Diagram of Gait State Machine.
Table 1 Support categories and their respective conditions. Support state
Pressure condition
NoSupport (NS) SitSupport (SitS) DoubleSupport (DS) SingleSupport (SS)
P < Silo Silo ࣘ P < DSlo DSlo ࣘ P < SSlo P > DSlo
most other wearable fall sensors, as the concept uses knowledge on the current physical activity undertaken by the user and does not rely on impact to identify falls, the sensor is capable of detecting so called sliding falls that are hard to detect with impact based fall sensors. 2. Principle of operation Pressure exerted by the foot and orientation of the foot (determined from acceleration) are gathered periodically and used in a state machine to determine the current physical activity undertaken by the user. With the knowledge captured in the state machine, fall events can be detected as abnormal changes in pressure and acceleration in a given state. In such a case, the fall sensor reports the fall event to a fall handler, which typically resides on a mobile phone. The fall handler can then take appropriate action, such as informing a formal or informal caretaker, or emergency services. The fall algorithm was developed as two separate state machines. The first state machine is called the Gait state machine. This state machine has two states: Static and Dynamic. State transitions are controlled by changes in measured pressure, P, and through the use of time outs. The measured pressure is related to a Support State as listed in Table 1, where NoSupport (NS) is equivalent to the sensor being suspended in air, SitSupport (SitS) indicates a low but nonzero pressure related to the user sitting and Double Support (DS) and SingleSupport (SS) relate to both feet and one foot respectively being in contact with the ground. The thresholds used to distinguish between these support states are:
•
•
•
Silo indicates the upper threshold pressure for a foot in the air and is also used as the lower threshold pressure experienced while sitting. DSlo indicates the lower threshold pressure for two feet on the ground while standing. SSlo indicates the lower threshold pressure experienced when standing on one foot.
State transitions in the Gait state machine are governed by parameters identified in the Gait state machine. If the current state is Static and a pressure lower than the Silo threshold is measured, the foot is assumed to be taken off the ground and the state becomes Dynamic. Upon a pressure reduction below DSlo , or upon a timer running out, the state will change from Dynamic to Static. A repeated transition between these two states within certain time limits results in a non-zero gait period, support time and swing time. The Gait state machine is depicted in Fig. 1. For brevity of the transition conditions, the numbers next to the state transitions indicate the priority of rules in case more than one transition from the current state is possible at a given time. 1. The three rules associated with the Static state are as follows: If the measured pressure is lower than Silo , the state changes from Static to Dynamic. In addition to this state change, the time spent in the Static state (Tstat ) is recorded as the support time, Tsupp . Furthermore the variable Pmin , which keeps track of the minimum pressure measured by the sensor, is reset. 2. If the time spent in the Static state exceeds a threshold, it is concluded that no gait pattern is available. α , the orientation of the foot relative to gravity is recorded and the Activity state machine is called. 3. If neither rule 1 nor rule 2 applies, the Static state remains active and P¯ stat , the average pressure measured in the static state, and the maximum pressure measured, Pmax , are updated frequently. The two rules associated with the Dynamic state are as follows: 1. If the measured pressure is greater than the threshold DSlo plus the minimum pressure measured in the Dynamic state, the latter
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
501
Table 2 Formulae for limits used in determining weight on foot. Parameter
Description
Condition
Pmax
Maximum pressure per cycle while conditions hold
P > DSlo
Pmax
Average maximum pressure over all cycles
Pstat
Average pressure per cycle while state = ‘Static’.
P > DSlo
Pdyn
Average pressure per cycle while state = ‘Dynamic’.
P < DSlo
Formula Pmax = maxn P (n) Pmax (n) = Pmax (n − 1) n−1 + n
Pmax n
Pstat (k) = Pstat (k − 1) k−1 + P(kk) k Pdyn (l) = Pdyn (l − 1) l−1 + P(ll) l
Table 3 Pressure related thresholds. Parameter
Description
Formula
SSlo
Lowest pressure value that indicates single support
SSlo (n) = SSlo (n − 1) n−1 + n
DSlo
Lowest pressure value that indicates double support
DSlo (n) = DSlo (n − )
Silo
Lowest pressure value that indicates sit support
Silo (n) = Silo (n − 1)
changes to Static. This state change will also occur if the time that the Dynamic state has been active continuously, Tdyn , exceeds the max higher threshold Tswing . In addition to the state change, a number of book keeping procedures is performed: a. The swing time for the current gait cycle (Tswing ) is set to the time the algorithm remained in the Dynamic state. b. The gait-cycle-average of minimum pressure, P¯ min is updated. c. Pressure parameters and pressure thresholds listed in Table 2 and Table 3 respectively are updated. d. The sensor’s orientation α is updated. e. The activity state machine is updated. f. Tstat , Tdyn , the gait-cycle-averages for pressure measured in the Static and the Dynamic state (P¯ stat and P¯ dyn respectively) and Pmax are reset. The state machine contains a number of user-specific pressure (and thus weight) related parameters as listed in Table 2. Pmax is the maximum pressure in a given cycle and is used to calculate the average maximum pressure Pmax recursively. Similarly, the average pressure Pstat and Pdyn exerted while the Gait state machine is in respectively the Static state or the Dynamic state is calculated as a recursive average. The latter two, being cycle averages, are reset with every Gait machine state change and hence so are the counters containing parameters k and l. The parameter n is not reset and thus represents an averaging effect from time 0 onwards where time 0 represents a hard reset of the system. The obtained pressure statistics are used to estimate threshold parameters between the NoSupport and the SitSupport state (Silo ), between the SitSupport and Double Support state (DSlo ) and between the Double Support and Single Support state (SSlo ) as listed in Table 3. SSlo is calculated as the recursive time average of the average of Pmax and Pmax . The averaging of these two pressure statistics represents a weighted average variant of Pmax which puts a higher emphasis on currently measured Pmax. DSlo is calculated as the recursive time average of the average of Pstat , Pdyn and Pmax . The average of Pstat and Pdyn can be interpreted as the current approximation of Double Support pressure. The addition of Pmax to this average adds a lag to the estimate which aids temporal stability and dampens sudden changes. Silo , as an indicator of highest pressure in the static state, is approximated using the recursive time average of the per cycle average pressure in the dynamic state. With these threshold parameters, changes in support are tracked and used in the Activity State Machine, which is depicted in Fig. 2 and continuously tracks the user’s physical activity in terms of 7 states (Lie, Sit, Stand, Walk, Run, FALL and SitTransit). State transitions are governed by the currently active support state (NS, SitS, DS or SS), the
Pmax +Pmax 2n P +Pdyn +Pmax 1 + stat 3n P n−1 + dyn n n n−1 n
sensor’s orientation relative to ground, α , and time spent in a state, T. The used thresholds are: •
•
The time-outs governing transitions between the Stand, Walk and max , where state can be Walk or Run state are both of the form Tstate Run. These time-outs are used to trigger a state machine cycle. αlie is the angle relative to gravity associated with the user being in a lying position.
Operators used are of Boolean type (logical OR: | and logical AND: &) and support states equate to TRUE if they are currently active. Hardware for the fall sensor was custom made with a microprocessor, resistive pressure sensors, off-the-shelf Bluetooth module and discrete accelerometer. As the accelerometer is used to measure orientation of the shoe and not to measure a high impact, the acceleration measurement can be performed at a frequency below 10 Hz. The accelerometer employed (FreeScale MMA7361L) features a power consumption of 400 μA in normal operation which reduces to 3 μA in sleep mode. The microprocessor (Texas Instruments MSP430F1611) uses 330 μA at 1 MHz clock frequency and 2.2 V power supply which reduces to 1.1 μA and 0.2 μA in standby and off mode respectively. The Bluetooth module (Roving Networks RN41) is the highest consumer with a 30 mA power consumption while connected. However, the duty cycle of Bluetooth messages is relatively low and hence the power consumption of the Bluetooth module does not pose insurmountable restrictions. The circuitry with sensors, microprocessor, Bluetooth module and 1100 mAh battery was mounted on a shoe as shown in Fig. S3. A strap wrapped around the front of the subject’s shoe is the mounting point for the processor board, Bluetooth module and power source. Sensors are integrated in straps that run under the sole of the shoe. Straps on either side of the shoe allow for resizing of the sensor system and act as conduit for the wires coming from sensors under the heel of the shoe. Software was developed in C (for the fall sensor) and Java (for the Blackberry). In the following sections the fall and mobility sensor will be referred to as the ‘fall sensor’ whereas the application running on the mobile device will be referred to as the ‘fall handler’. The fall sensor software consists of a set of interrupt routines. At fixed intervals, an Analogue-Digital-Converter reading is taken from the pressure sensors mounted under the user’s foot and the accelerometer indicating orientation of the foot. These readings are used to update the two state machines presented previously. If event or alert messages result from this analysis, these are stored on a stack and a transmission will be initiated. Communication between fall sensor and fall handler is accommodated by a Universal Asynchronous Receiver/Transmitter interface. Sensor and user status are communicated to the fall handler, which runs on a Blackberry mobile phone. The fall handler interprets the
502
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
Fig. 2. Activity State Machine.
received messages and allows for appropriate action to be taken. Such action would typically take the form of a warning to the user which, if not cancelled by the user, results in a voice call to a formal or informal caretaker, or the emergency services. 3. Proof of concept tests In a functional verification of the proposed fall sensor concept, the sensor was used to demonstrate training of the various parameters, identification of physical activities and identification of a sliding fall. Training of the various user-specific parameters is done automatically without the need for extraordinary user actions. The algorithms use normal walking for parameter estimation and automatically extract the pressure exerted on the sensor for the four support conditions (SS, DS, SitS, NS). This is demonstrated in Fig. 3. From 0 to 65 s into this test, the user performed the following sequence: 1. Stand on one leg (resulting in support state SingleSupport (SS)) 2. Stand on two legs (resulting in support state DoubleSupport (DS)) 3. Sit (resulting in support state SitSupport (SitS))
4. Foot in the air (resulting in support state NoSupport (NS)) 5. Stand on two legs (again resulting in support state DoubleSupport) The activity as inferred by the Activity state machine is indicated with double arrows at the top of the figure, each with the inferred activity directly below the arrow. It can be seen that at the beginning of the experiment, the identification of the activities is incorrect as the weight parameters have not yet been identified. Once the user starts walking, the algorithms start the training and it can indeed be seen that after about 70 s of normal walking the curves are flattening off, indicating that the end of training has been reached. After this period, the user performs the same set of activities and now it can be seen that the activity is correctly identified. SingleSupport and DoubleSupport support states are correctly identified as the activity ‘Stand’ whereas the support states SitSupport and NoSupport are identified as the activity ‘Sit’. Note also that the foot off the ground event is correctly identified as sitting and not as lying. For this the accelerometers are used to determine the orientation of the user’s shoe.
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
503
Fig. 3. Weight distribution curves while training.
Fig. 4. Experiment to identify various activities.
It should also be noted that, even though in the tests here presented training is performed at the beginning of the test, this is not necessary during normal use. The sensor stores the obtained values but will be able to change these rapidly if necessary. In a second test the algorithm’s capabilities of identifying normal activities of daily living are demonstrated. In this test, which is represented graphically in Fig. 4, the subject started off walking. At 68 s, the subject sat down on a couch and then went from a sitting to a lying position on the same couch starting at 123 s. At 182 s into the experiment the subject stood up from the couch and started walking for 60 s after which the subject immediately went to a lying position on a couch for 60 s, followed by a 30 s period of sitting. After this the subject stood up again from the couch and walked for another 80 s. Immediately following the last period of walking, the subject performed the following actions: 1. 2. 3. 4.
Stand on one leg (from 400 s to 408 s) Stand on two legs (from 408 s to 416 s) Sit (from 416 s to 424 s) Sit with foot in the air (from 424 s to 432 s)
Fig. 4 shows that after the initial 68 s of training the parameters were sufficiently accurately identified to detect all successive activities accurately. This figure also shows how identification is a continuous process; after 68 s the parameters (in particular SSlo had not been fully identified. Further identification is performed during the second bout of walking (from 182 to 242 s) and the limited changes to these parameters in the third bout of walking (from 332 to 412 s) show that the training process has come close to a steady state. The final test in this section demonstrates the sensor’s ability to identify fall events. To demonstrate the capabilities of the developed sensor over impact based fall sensors a so-called ‘sliding fall’ was used in this test. In a sliding fall the person loses balance but manages to relatively slowly hit the ground, perhaps due to the vicinity of a wall or hand railing or due to the support of a peer. These falls are commonly missed by commercial sensors due to the absence of abrupt changes in the parameters measured. Fig. 5 shows the test results. After a brief static period, the test subject started walking for 60 s after which the fall occurred. The sliding fall was performed against a wall and the subject ended up in a more or less sitting position against the wall. Nevertheless the fall was correctly identified. After the fall, the subject
504
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
Fig. 5. Experiment showing identification of a sliding fall.
stood up and started walking again. Also this is correctly identified by the sensor. 4. Conclusions This paper describes the development and functional testing of a fall sensor based on the measurement of acceleration and pressure in a non-traditional way. The information from these sensors is used to update a state-machine that keeps track of the user’s physical activity state, from which fall incidents can be inferred. The pressure sensor acts as an information source for locomotion but also static states. Further information is obtained from an accelerometer that is used to measure the orientation of the user’s foot and thus allows identifying orientations that are not associated with normal activities. Tests with the device show that it accurately detects both activities of daily living and fall events. In particular, the so called sliding fall, which often remains undetected when using more traditional impact based monitors, was accurately detected. The developed sensor offers various advantages over commercially available solutions. The self-training capabilities of the sensor have been demonstrated and this feature allows for accurate, individually tailored operation. Although the gait and activity state machines have been demonstrated to be able to identify a number of activities being performed, this list of activities is limited. To extend the set of identified activities with the current approach may prove cumbersome as the set of parameters used to formulate the transition conditions in the state machines may not be rich enough. Improvements could be obtained through the use of classifiers such as support vector machines, decision trees or back propagation neural networks. With an extensive dataset such algorithms could be trained to recognise a wider variety of activities of daily living. This information in turn would provide valuable information on general activity and mobility of the user and could thus be used to analyse the fall risk of users prior to the user having experienced a fall. It should also be noted that to date only functional tests have been performed with the sensor. Future work will focus on a validation of the sensor with both healthy subjects and a representative group of elderly users. Stigmatisation is of considerable concern in providing effective and acceptable solutions for falls detection. The sensor’s proposed location in footwear is an important factor in improving acceptance of the sensor by the main target group of elderly citizens. Although the present sensor consists of off-the-shelf components which prevent effective miniaturisation in its current form, other research endeavours have demonstrated that integration in an insole of such systems is feasible.
Active management of one’s health is of increasing interest. As the sensor is equipped with a Bluetooth interface, it can communicate with 3rd party devices such as the user’s mobile phone. Hence, the user’s phone can not only act as an intermediate between the fall sensor and a formal or informal caretaker, but can also be used to provide the user with pertinent information on their mobility patterns. Such information may result in improved awareness of the user in their own mobility and activity patterns and can thus help the user to adapt a healthier lifestyle. Funding source This work was funded by Enterprise Ireland under their Commercialisation Fund, PC 2008 0179. Ethical approval Ethical approval for this study was granted by the University of Limerick Research Ethics Governance Committee. Supplementary Materials Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.medengphy.2015.02.009. Conflict of interest None declared. References [1] Rubenstein LZ. Falls in older people: epidemiology, risk factors and strategies for prevention. Age and Ageing 2006;35:37–41. [2] Health Service Authority. Strategy to prevent falls and fractures in Ireland’s ageing population. Dublin: National Council on Ageing and Older People, Department of Health and Children, Health Service Executive; 2008. [3] Lord S, Sherrington C, Hylton M, Close J. Falls in older people: risk factors and strategies for prevention. 2nd ed. Cambridge University Press; 2007. [4] Tinetti ME, Claus EB. Predictors and prognosis of inability to get up after falls among elderly persons. J Am Med Assoc 1993;269:65–70. [5] Skelton DA. Effects of physical activity on postural stability. Age and Ageing 2001;30:33–9. [6] Wild DN, Iaacs B. How dangerous are falls in old people at home. Br Med J (Clin Res) 1981;282:266–8. [7] Kang HG, Mahoney DF, Hoenig H, Hirth VA, Bonato P, Hajjar I, et al. In situ monitoring of health in older adults: technologies and issues. J Am Geriatr Soc 2010;58:1579–86. [8] Johannes Oberzaucher HJ, Zödl C, Hlauschek W, Zagle W, Using a wearable insole gait analyzing system for automated mobility assessment for older people. In: Miesenberger K, Zagler W, Karshmer A, editors. Computers helping people with special needs. Springer Verlag; 2010 [9] Gupta P. Real-time fall detection system using wireless MEMS sensors and ZigBee protocol. Texas Tech University; 2009.
Contents lists available at ScienceDirect
Medical Engineering and Physics journal homepage: www.elsevier.com/locate/medengphy
Technical note
Unobtrusive monitoring and identification of fall accidents Pepijn van de Ven a,∗, Hugh O’Brien a, John Nelson a, Amanda Clifford b a b
Department of Electronic and Computer Engineering, University of Limerick, Limerick, Ireland Department of Clinical Therapies, University of Limerick, Limerick, Ireland
a r t i c l e
i n f o
Article history: Received 3 October 2013 Revised 18 December 2014 Accepted 16 February 2015
Keywords: Fall sensing Falls prevention Accelerometry Ambient assisted living Mobile health
a b s t r a c t Falls are a societal and economic problem of great concern with large parts of the population, in particular older citizens, at significant risk and the result of a fall often being grave. It has long been established that it is of importance to provide help to a faller soon after the event to prevent complications and this can be achieved with a fall monitor. Yet, the practical use of currently available fall monitoring solutions is limited due to accuracy, usability, cost, and, not in the least, the stigmatising effect of many solutions. This paper proposes a fall sensor concept that can be embedded in the user’s footwear and discusses algorithms, software and hardware developed. Sensor performance is illustrated using results of a series of functional tests. These show that the developed sensor can be used for the accurate measurement of various mobility and gait parameters and that falls are detected accurately. © 2015 IPEM. Published by Elsevier Ltd. All rights reserved.
1. Introduction In this paper a sensor proof of concept is presented that offers a solution to a range of issues encountered in traditional fall sensors. A third of older people over the age of 65 years fall in the community each year with prevalence rates increasing to over a half of individuals over 80 years [1]. Falls related injuries come at a high cost to society and individuals. Society as a whole is faced with high annual bills in the order of €400 million for a country such as Ireland and this is predicted to increase to €1 billion by 2020 [2]. Following a fall, individuals may be impeded in their daily lives by the fear of falling, reduced self-efficacy and the physical consequences of the fall [3]. A high proportion of uninjured fallers (47%) are unable to get up from the floor after a fall [4] due to lack of strength and physical fitness [5]. An inability to get up from the floor after a fall can result in a long lie, which is defined as remaining on the floor for more than an hour after a fall. The long lie is associated with co-morbidities including dehydration, pneumonia, hypothermia and pressure sores and has been found to significantly increase the probability of death (50%) within 6 months [3,6]. Hence, detection of a fall is of great importance and numerous fall detection systems have been developed. Most wearable fall detection systems rely on detecting impact. This, however, is not always a feature of significance in a fall. Moreover, device discreteness and obtrusiveness negatively impact on the acceptability and usability of many current devices [7].
∗
Corresponding author. Tel.: +353 61202925. E-mail address: [email protected] (P. van de Ven).
http://dx.doi.org/10.1016/j.medengphy.2015.02.009 1350-4533/© 2015 IPEM. Published by Elsevier Ltd. All rights reserved.
This paper presents a sensor concept that offers a solution to these issues. The presented sensor is a footwear based sensor that uses pressure, acceleration and temporal information to assess the user’s ambulatory parameters and the incidence of falls. The use of footwear based sensors has been investigated by various groups [8,9].1,2 However, the main focus of these efforts has been on falls prevention through monitoring of gait and mobility parameters. Although falls risk assessment is an important aspect of the management of falls, it will never be possible to fully rule out the occurrence of a fall. Hence, the detection of the fall event itself will remain of crucial importance in any falls emergency system. The focus on falls prevention for footwear based sensors is likely for more than one reason. As the old proverb says “prevention is better than cure”, but a likely further reason for the focus on falls prediction for footwear based sensors is that in this location a fall event is difficult to identify with the classical fall detection approach of measuring acceleration and impact. In this work we propose methods to obtain accurate fall detection from the user’s foot by using a state machine which allows the measured pressure and acceleration to be assessed within the context of the current user physical activity. This strategy allows for a relatively low sampling frequency, which in turn results in lower battery consumption. The sensor would be relatively cheap to manufacture and as the concept allows for a solution fully embedded in the user’s footwear, is fully unobtrusive. Moreover, unlike 1 WIISEL – Wireless Insole for Independent and Safe Elderly Living (http://www.wiisel.eu/). 2 Balance Problems? Step into the iShoe (http://web.mit.edu/newsoffice/2008/ i-shoe-0716.html).
500
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
Fig. 1. Diagram of Gait State Machine.
Table 1 Support categories and their respective conditions. Support state
Pressure condition
NoSupport (NS) SitSupport (SitS) DoubleSupport (DS) SingleSupport (SS)
P < Silo Silo ࣘ P < DSlo DSlo ࣘ P < SSlo P > DSlo
most other wearable fall sensors, as the concept uses knowledge on the current physical activity undertaken by the user and does not rely on impact to identify falls, the sensor is capable of detecting so called sliding falls that are hard to detect with impact based fall sensors. 2. Principle of operation Pressure exerted by the foot and orientation of the foot (determined from acceleration) are gathered periodically and used in a state machine to determine the current physical activity undertaken by the user. With the knowledge captured in the state machine, fall events can be detected as abnormal changes in pressure and acceleration in a given state. In such a case, the fall sensor reports the fall event to a fall handler, which typically resides on a mobile phone. The fall handler can then take appropriate action, such as informing a formal or informal caretaker, or emergency services. The fall algorithm was developed as two separate state machines. The first state machine is called the Gait state machine. This state machine has two states: Static and Dynamic. State transitions are controlled by changes in measured pressure, P, and through the use of time outs. The measured pressure is related to a Support State as listed in Table 1, where NoSupport (NS) is equivalent to the sensor being suspended in air, SitSupport (SitS) indicates a low but nonzero pressure related to the user sitting and Double Support (DS) and SingleSupport (SS) relate to both feet and one foot respectively being in contact with the ground. The thresholds used to distinguish between these support states are:
•
•
•
Silo indicates the upper threshold pressure for a foot in the air and is also used as the lower threshold pressure experienced while sitting. DSlo indicates the lower threshold pressure for two feet on the ground while standing. SSlo indicates the lower threshold pressure experienced when standing on one foot.
State transitions in the Gait state machine are governed by parameters identified in the Gait state machine. If the current state is Static and a pressure lower than the Silo threshold is measured, the foot is assumed to be taken off the ground and the state becomes Dynamic. Upon a pressure reduction below DSlo , or upon a timer running out, the state will change from Dynamic to Static. A repeated transition between these two states within certain time limits results in a non-zero gait period, support time and swing time. The Gait state machine is depicted in Fig. 1. For brevity of the transition conditions, the numbers next to the state transitions indicate the priority of rules in case more than one transition from the current state is possible at a given time. 1. The three rules associated with the Static state are as follows: If the measured pressure is lower than Silo , the state changes from Static to Dynamic. In addition to this state change, the time spent in the Static state (Tstat ) is recorded as the support time, Tsupp . Furthermore the variable Pmin , which keeps track of the minimum pressure measured by the sensor, is reset. 2. If the time spent in the Static state exceeds a threshold, it is concluded that no gait pattern is available. α , the orientation of the foot relative to gravity is recorded and the Activity state machine is called. 3. If neither rule 1 nor rule 2 applies, the Static state remains active and P¯ stat , the average pressure measured in the static state, and the maximum pressure measured, Pmax , are updated frequently. The two rules associated with the Dynamic state are as follows: 1. If the measured pressure is greater than the threshold DSlo plus the minimum pressure measured in the Dynamic state, the latter
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
501
Table 2 Formulae for limits used in determining weight on foot. Parameter
Description
Condition
Pmax
Maximum pressure per cycle while conditions hold
P > DSlo
Pmax
Average maximum pressure over all cycles
Pstat
Average pressure per cycle while state = ‘Static’.
P > DSlo
Pdyn
Average pressure per cycle while state = ‘Dynamic’.
P < DSlo
Formula Pmax = maxn P (n) Pmax (n) = Pmax (n − 1) n−1 + n
Pmax n
Pstat (k) = Pstat (k − 1) k−1 + P(kk) k Pdyn (l) = Pdyn (l − 1) l−1 + P(ll) l
Table 3 Pressure related thresholds. Parameter
Description
Formula
SSlo
Lowest pressure value that indicates single support
SSlo (n) = SSlo (n − 1) n−1 + n
DSlo
Lowest pressure value that indicates double support
DSlo (n) = DSlo (n − )
Silo
Lowest pressure value that indicates sit support
Silo (n) = Silo (n − 1)
changes to Static. This state change will also occur if the time that the Dynamic state has been active continuously, Tdyn , exceeds the max higher threshold Tswing . In addition to the state change, a number of book keeping procedures is performed: a. The swing time for the current gait cycle (Tswing ) is set to the time the algorithm remained in the Dynamic state. b. The gait-cycle-average of minimum pressure, P¯ min is updated. c. Pressure parameters and pressure thresholds listed in Table 2 and Table 3 respectively are updated. d. The sensor’s orientation α is updated. e. The activity state machine is updated. f. Tstat , Tdyn , the gait-cycle-averages for pressure measured in the Static and the Dynamic state (P¯ stat and P¯ dyn respectively) and Pmax are reset. The state machine contains a number of user-specific pressure (and thus weight) related parameters as listed in Table 2. Pmax is the maximum pressure in a given cycle and is used to calculate the average maximum pressure Pmax recursively. Similarly, the average pressure Pstat and Pdyn exerted while the Gait state machine is in respectively the Static state or the Dynamic state is calculated as a recursive average. The latter two, being cycle averages, are reset with every Gait machine state change and hence so are the counters containing parameters k and l. The parameter n is not reset and thus represents an averaging effect from time 0 onwards where time 0 represents a hard reset of the system. The obtained pressure statistics are used to estimate threshold parameters between the NoSupport and the SitSupport state (Silo ), between the SitSupport and Double Support state (DSlo ) and between the Double Support and Single Support state (SSlo ) as listed in Table 3. SSlo is calculated as the recursive time average of the average of Pmax and Pmax . The averaging of these two pressure statistics represents a weighted average variant of Pmax which puts a higher emphasis on currently measured Pmax. DSlo is calculated as the recursive time average of the average of Pstat , Pdyn and Pmax . The average of Pstat and Pdyn can be interpreted as the current approximation of Double Support pressure. The addition of Pmax to this average adds a lag to the estimate which aids temporal stability and dampens sudden changes. Silo , as an indicator of highest pressure in the static state, is approximated using the recursive time average of the per cycle average pressure in the dynamic state. With these threshold parameters, changes in support are tracked and used in the Activity State Machine, which is depicted in Fig. 2 and continuously tracks the user’s physical activity in terms of 7 states (Lie, Sit, Stand, Walk, Run, FALL and SitTransit). State transitions are governed by the currently active support state (NS, SitS, DS or SS), the
Pmax +Pmax 2n P +Pdyn +Pmax 1 + stat 3n P n−1 + dyn n n n−1 n
sensor’s orientation relative to ground, α , and time spent in a state, T. The used thresholds are: •
•
The time-outs governing transitions between the Stand, Walk and max , where state can be Walk or Run state are both of the form Tstate Run. These time-outs are used to trigger a state machine cycle. αlie is the angle relative to gravity associated with the user being in a lying position.
Operators used are of Boolean type (logical OR: | and logical AND: &) and support states equate to TRUE if they are currently active. Hardware for the fall sensor was custom made with a microprocessor, resistive pressure sensors, off-the-shelf Bluetooth module and discrete accelerometer. As the accelerometer is used to measure orientation of the shoe and not to measure a high impact, the acceleration measurement can be performed at a frequency below 10 Hz. The accelerometer employed (FreeScale MMA7361L) features a power consumption of 400 μA in normal operation which reduces to 3 μA in sleep mode. The microprocessor (Texas Instruments MSP430F1611) uses 330 μA at 1 MHz clock frequency and 2.2 V power supply which reduces to 1.1 μA and 0.2 μA in standby and off mode respectively. The Bluetooth module (Roving Networks RN41) is the highest consumer with a 30 mA power consumption while connected. However, the duty cycle of Bluetooth messages is relatively low and hence the power consumption of the Bluetooth module does not pose insurmountable restrictions. The circuitry with sensors, microprocessor, Bluetooth module and 1100 mAh battery was mounted on a shoe as shown in Fig. S3. A strap wrapped around the front of the subject’s shoe is the mounting point for the processor board, Bluetooth module and power source. Sensors are integrated in straps that run under the sole of the shoe. Straps on either side of the shoe allow for resizing of the sensor system and act as conduit for the wires coming from sensors under the heel of the shoe. Software was developed in C (for the fall sensor) and Java (for the Blackberry). In the following sections the fall and mobility sensor will be referred to as the ‘fall sensor’ whereas the application running on the mobile device will be referred to as the ‘fall handler’. The fall sensor software consists of a set of interrupt routines. At fixed intervals, an Analogue-Digital-Converter reading is taken from the pressure sensors mounted under the user’s foot and the accelerometer indicating orientation of the foot. These readings are used to update the two state machines presented previously. If event or alert messages result from this analysis, these are stored on a stack and a transmission will be initiated. Communication between fall sensor and fall handler is accommodated by a Universal Asynchronous Receiver/Transmitter interface. Sensor and user status are communicated to the fall handler, which runs on a Blackberry mobile phone. The fall handler interprets the
502
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
Fig. 2. Activity State Machine.
received messages and allows for appropriate action to be taken. Such action would typically take the form of a warning to the user which, if not cancelled by the user, results in a voice call to a formal or informal caretaker, or the emergency services. 3. Proof of concept tests In a functional verification of the proposed fall sensor concept, the sensor was used to demonstrate training of the various parameters, identification of physical activities and identification of a sliding fall. Training of the various user-specific parameters is done automatically without the need for extraordinary user actions. The algorithms use normal walking for parameter estimation and automatically extract the pressure exerted on the sensor for the four support conditions (SS, DS, SitS, NS). This is demonstrated in Fig. 3. From 0 to 65 s into this test, the user performed the following sequence: 1. Stand on one leg (resulting in support state SingleSupport (SS)) 2. Stand on two legs (resulting in support state DoubleSupport (DS)) 3. Sit (resulting in support state SitSupport (SitS))
4. Foot in the air (resulting in support state NoSupport (NS)) 5. Stand on two legs (again resulting in support state DoubleSupport) The activity as inferred by the Activity state machine is indicated with double arrows at the top of the figure, each with the inferred activity directly below the arrow. It can be seen that at the beginning of the experiment, the identification of the activities is incorrect as the weight parameters have not yet been identified. Once the user starts walking, the algorithms start the training and it can indeed be seen that after about 70 s of normal walking the curves are flattening off, indicating that the end of training has been reached. After this period, the user performs the same set of activities and now it can be seen that the activity is correctly identified. SingleSupport and DoubleSupport support states are correctly identified as the activity ‘Stand’ whereas the support states SitSupport and NoSupport are identified as the activity ‘Sit’. Note also that the foot off the ground event is correctly identified as sitting and not as lying. For this the accelerometers are used to determine the orientation of the user’s shoe.
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
503
Fig. 3. Weight distribution curves while training.
Fig. 4. Experiment to identify various activities.
It should also be noted that, even though in the tests here presented training is performed at the beginning of the test, this is not necessary during normal use. The sensor stores the obtained values but will be able to change these rapidly if necessary. In a second test the algorithm’s capabilities of identifying normal activities of daily living are demonstrated. In this test, which is represented graphically in Fig. 4, the subject started off walking. At 68 s, the subject sat down on a couch and then went from a sitting to a lying position on the same couch starting at 123 s. At 182 s into the experiment the subject stood up from the couch and started walking for 60 s after which the subject immediately went to a lying position on a couch for 60 s, followed by a 30 s period of sitting. After this the subject stood up again from the couch and walked for another 80 s. Immediately following the last period of walking, the subject performed the following actions: 1. 2. 3. 4.
Stand on one leg (from 400 s to 408 s) Stand on two legs (from 408 s to 416 s) Sit (from 416 s to 424 s) Sit with foot in the air (from 424 s to 432 s)
Fig. 4 shows that after the initial 68 s of training the parameters were sufficiently accurately identified to detect all successive activities accurately. This figure also shows how identification is a continuous process; after 68 s the parameters (in particular SSlo had not been fully identified. Further identification is performed during the second bout of walking (from 182 to 242 s) and the limited changes to these parameters in the third bout of walking (from 332 to 412 s) show that the training process has come close to a steady state. The final test in this section demonstrates the sensor’s ability to identify fall events. To demonstrate the capabilities of the developed sensor over impact based fall sensors a so-called ‘sliding fall’ was used in this test. In a sliding fall the person loses balance but manages to relatively slowly hit the ground, perhaps due to the vicinity of a wall or hand railing or due to the support of a peer. These falls are commonly missed by commercial sensors due to the absence of abrupt changes in the parameters measured. Fig. 5 shows the test results. After a brief static period, the test subject started walking for 60 s after which the fall occurred. The sliding fall was performed against a wall and the subject ended up in a more or less sitting position against the wall. Nevertheless the fall was correctly identified. After the fall, the subject
504
P. van de Ven et al. / Medical Engineering and Physics 37 (2015) 499–504
Fig. 5. Experiment showing identification of a sliding fall.
stood up and started walking again. Also this is correctly identified by the sensor. 4. Conclusions This paper describes the development and functional testing of a fall sensor based on the measurement of acceleration and pressure in a non-traditional way. The information from these sensors is used to update a state-machine that keeps track of the user’s physical activity state, from which fall incidents can be inferred. The pressure sensor acts as an information source for locomotion but also static states. Further information is obtained from an accelerometer that is used to measure the orientation of the user’s foot and thus allows identifying orientations that are not associated with normal activities. Tests with the device show that it accurately detects both activities of daily living and fall events. In particular, the so called sliding fall, which often remains undetected when using more traditional impact based monitors, was accurately detected. The developed sensor offers various advantages over commercially available solutions. The self-training capabilities of the sensor have been demonstrated and this feature allows for accurate, individually tailored operation. Although the gait and activity state machines have been demonstrated to be able to identify a number of activities being performed, this list of activities is limited. To extend the set of identified activities with the current approach may prove cumbersome as the set of parameters used to formulate the transition conditions in the state machines may not be rich enough. Improvements could be obtained through the use of classifiers such as support vector machines, decision trees or back propagation neural networks. With an extensive dataset such algorithms could be trained to recognise a wider variety of activities of daily living. This information in turn would provide valuable information on general activity and mobility of the user and could thus be used to analyse the fall risk of users prior to the user having experienced a fall. It should also be noted that to date only functional tests have been performed with the sensor. Future work will focus on a validation of the sensor with both healthy subjects and a representative group of elderly users. Stigmatisation is of considerable concern in providing effective and acceptable solutions for falls detection. The sensor’s proposed location in footwear is an important factor in improving acceptance of the sensor by the main target group of elderly citizens. Although the present sensor consists of off-the-shelf components which prevent effective miniaturisation in its current form, other research endeavours have demonstrated that integration in an insole of such systems is feasible.
Active management of one’s health is of increasing interest. As the sensor is equipped with a Bluetooth interface, it can communicate with 3rd party devices such as the user’s mobile phone. Hence, the user’s phone can not only act as an intermediate between the fall sensor and a formal or informal caretaker, but can also be used to provide the user with pertinent information on their mobility patterns. Such information may result in improved awareness of the user in their own mobility and activity patterns and can thus help the user to adapt a healthier lifestyle. Funding source This work was funded by Enterprise Ireland under their Commercialisation Fund, PC 2008 0179. Ethical approval Ethical approval for this study was granted by the University of Limerick Research Ethics Governance Committee. Supplementary Materials Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.medengphy.2015.02.009. Conflict of interest None declared. References [1] Rubenstein LZ. Falls in older people: epidemiology, risk factors and strategies for prevention. Age and Ageing 2006;35:37–41. [2] Health Service Authority. Strategy to prevent falls and fractures in Ireland’s ageing population. Dublin: National Council on Ageing and Older People, Department of Health and Children, Health Service Executive; 2008. [3] Lord S, Sherrington C, Hylton M, Close J. Falls in older people: risk factors and strategies for prevention. 2nd ed. Cambridge University Press; 2007. [4] Tinetti ME, Claus EB. Predictors and prognosis of inability to get up after falls among elderly persons. J Am Med Assoc 1993;269:65–70. [5] Skelton DA. Effects of physical activity on postural stability. Age and Ageing 2001;30:33–9. [6] Wild DN, Iaacs B. How dangerous are falls in old people at home. Br Med J (Clin Res) 1981;282:266–8. [7] Kang HG, Mahoney DF, Hoenig H, Hirth VA, Bonato P, Hajjar I, et al. In situ monitoring of health in older adults: technologies and issues. J Am Geriatr Soc 2010;58:1579–86. [8] Johannes Oberzaucher HJ, Zödl C, Hlauschek W, Zagle W, Using a wearable insole gait analyzing system for automated mobility assessment for older people. In: Miesenberger K, Zagler W, Karshmer A, editors. Computers helping people with special needs. Springer Verlag; 2010 [9] Gupta P. Real-time fall detection system using wireless MEMS sensors and ZigBee protocol. Texas Tech University; 2009.
