G Model JJBE-2429; No. of Pages 7
ARTICLE IN PRESS Medical Engineering & Physics xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Medical Engineering & Physics journal homepage: www.elsevier.com/locate/medengphy
Technical note
Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems Lei Gao ∗ , A.K. Bourke, John Nelson Department of Electronic and Computer Engineering, University of Limerick, Ireland
a r t i c l e
i n f o
Article history: Received 17 August 2012 Received in revised form 8 January 2014 Accepted 8 February 2014 Keywords: Activity recognition Multi-sensor fusion Accelerometer Activities of daily living
a b s t r a c t Physical activity has a positive impact on people’s well-being and it had been shown to decrease the occurrence of chronic diseases in the older adult population. To date, a substantial amount of research studies exist, which focus on activity recognition using inertial sensors. Many of these studies adopt a single sensor approach and focus on proposing novel features combined with complex classifiers to improve the overall recognition accuracy. In addition, the implementation of the advanced feature extraction algorithms and the complex classifiers exceed the computing ability of most current wearable sensor platforms. This paper proposes a method to adopt multiple sensors on distributed body locations to overcome this problem. The objective of the proposed system is to achieve higher recognition accuracy with “light-weight” signal processing algorithms, which run on a distributed computing based sensor system comprised of computationally efficient nodes. For analysing and evaluating the multi-sensor system, eight subjects were recruited to perform eight normal scripted activities in different life scenarios, each repeated three times. Thus a total of 192 activities were recorded resulting in 864 separate annotated activity states. The methods for designing such a multi-sensor system required consideration of the following: signal pre-processing algorithms, sampling rate, feature selection and classifier selection. Each has been investigated and the most appropriate approach is selected to achieve a trade-off between recognition accuracy and computing execution time. A comparison of six different systems, which employ single or multiple sensors, is presented. The experimental results illustrate that the proposed multi-sensor system can achieve an overall recognition accuracy of 96.4% by adopting the mean and variance features, using the Decision Tree classifier. The results demonstrate that elaborate classifiers and feature sets are not required to achieve high recognition accuracies on a multi-sensor system. © 2014 IPEM. Published by Elsevier Ltd. All rights reserved.
1. Introduction The undertaking of moderate to vigorous periods of activities of daily living (ADL) is now widely accepted to promote a healthier lifestyle in the older adult (>65 years) population [1,2]. Recently a significant amount of research has been performed into the application of wearable sensor-based systems for the measurement of the quantity and quality of the physical activities performed. Inertial sensors such as MEMS accelerometers and rate gyroscopes are now widely used for this application [3–5]. There has been significant work on activity recognition using a single sensor attached to different body locations [6–9]. In addition, recent advances in embedded sensor technology in smart phones,
∗ Corresponding author at: Department of Electronic and Computer Engineering, University of Limerick, Limerick, Ireland. Tel.: +353 87 7862690. E-mail address: [email protected] (L. Gao).
combined with their prevalence in modern day society, have promoted their application for the analysis of human movement activity classification [10,11]. However, three challenges primarily exist in these studies. • The recognition accuracy of the implementations on current single sensor platforms has not proven sufficiently accurate for ADL monitoring [12]. • Mobile phones by their nature are not fixed wearable sensors and inevitably periods of activity may be missed when characteristic functions are used (e.g. making a call). The carrying location of phones is often affected by the carrier’s gender (e.g. women may carry their phone in a hand-bag) and therefore the location also needs to be considered when designing the classification algorithm. • The algorithms for the long-term activity recognition with high recognition accuracy are limited by the trade-off between computational load requirements and battery life.
http://dx.doi.org/10.1016/j.medengphy.2014.02.012 1350-4533/© 2014 IPEM. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model JJBE-2429; No. of Pages 7
ARTICLE IN PRESS L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
2
Table 1 Scenario activities. Description Case 1 Case 2 Case 3 Case 4 Case 5 Case 6 Case 7 Case 8
Fig. 1. The data collection system with a garment involving four accelerometerbased sensors across the human body.
This paper thus proposes to use multiple sensors, executing computationally light-weight algorithms, on distributed body locations for more accurate and potentially more specific-activity recognition. The proposed method adopts the distributed computing concept, which will implement computationally inexpensive tasks on each sensor with the potential to achieve a higher degree of recognition accuracy. Thus the main contributions of this work can be summarized as follows: 1. The establishment of a case for multi-sensor systems through experimental observations: Based on the experimental results and previous studies, a wearable system using multiple sensors on different body locations is proposed, that considers both recognition accuracy and computing execution time as the relevant factors. 2. The comparison of wearable systems consisting of a single sensor or multiple sensors: The recognition accuracies of six wearable systems are compared under the same conditions. The objective is to compare two distinct types of wearable systems: single-sensor wearable systems adopting complex algorithms and multi-sensor systems employing light-weight algorithms. 2. Data collection Eight community dwelling older adults ranging in age from 70 to 83 (76.50 ± 4.41 years), capable of walking unaided and having various conditions such as osteoporosis, COPD, leg ulcer and knee replacement were recruited. Subjects were each fitted with four accelerometer based sensors, measuring accelerations at the chest, left under-arm, waist and thigh, as illustrated in Fig. 1. Subjects were fitted with a garment, to which two sensors were attached at the chest (sternum) and left under-arm. A sensor was attached at the waist (right anterior iliac crest of the pelvis) using a custom carry-case and waist-belt. Subjects gave informed consent to participate. The fourth sensor was worn on the thigh (in a pocket), contained within a random oriented padded box (98 mm × 42 mm × 27 mm). This arrangement simulated the shape and size of a smart phone in a protective case. The ShimmerTM wireless sensor platform [13] was used to record the raw triaxial accelerometer data. ShimmerTM is a small sensor platform (53 mm × 32 mm × 25 mm and 22 g) well suited for wearable applications. It includes an on-board tri-axial accelerometer, with a
Sitting down and standing up from an arm chair Sitting down and standing up from a kitchen chair Sitting down and standing up from a toilet seat Walking up and down stairs Sitting down and standing up from a bed Lying down and getting up from a bed Getting in and out of a car seat Walking 10 m
configurable sampling rate up to 1 kHz and an amplitude range up to ±6G. Each subject was asked to perform eight scenario activities, as listed in Table 1, each repeated three times. These were recorded in the subjects’ own home environment. Thus a total of 24 scenarios were recorded from each subject. The sitting-down and standingup activities were recorded in five different life scenarios, so the data set could be used to evaluate the performance of the wearable system in this representative real-life environment. The tri-axial accelerometer data was sampled at 200 Hz and at a resolution of 12 bits, from each of the four sensors and simultaneously transmitted to a laptop computer via a Bluetooth wireless body area network (WBAN). The harvested dataset was automatically annotated during acquisition on the computer. Each scenario commenced and concluded with the subject in a standing position for 5 s, thus 2 transitions were recorded in each scenario (standing-transitionactivity and activity-transition-standing) except for the walking scenarios. The walking scenarios were recorded according to the pattern (standing-walking-standing). These signals were then manually segmented into separate activity states, which belong to the categories listed in Table 2 during post-trial data processing and analysis. There are three categories for the target states: static activities, dynamic activities and transitional activities. Static activities, such as standing, are where a posture is held for a period of time. Dynamic activities involve physical movement, such as walking. Transitional activities were restricted to changing between static activities. In this study, the requirement is to recognize the five activity states listed in Table 2: standing, sitting, lying, walking and transition. The transition activities were grouped and not separately distinguished. The walking activity included both over ground walking and up and down stairs, which were not distinguished. Thus the six sitting and lying scenarios contained five states and the two walking scenarios contained three states. Each scenario was repeated three times, thereby contributing to a total of 108 states per subject. An example of the signal segmentation and annotation can be seen in Fig. 2.
Table 2 Separate activity states. State
Activity
Static activities
Lying Sitting Standing
Dynamic activities
Walking (flat walking and up & down stairs)
Transitional activities
Lying–Standing Standing–Lying Sitting–Standing Standing–Sitting
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model JJBE-2429; No. of Pages 7
ARTICLE IN PRESS L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
3
Fig. 2. The process of dividing and annotating a stand-sit-stand scenario is shown. It is (A) standing, (B) transition, (C) sitting, (D) transition, and (E) standing (1G = 9.81 m/s2 ).
3. Methods In this section, the design of a multi-sensor system is proposed and analyzed. The main steps can be categorized as signal preprocessing, feature extraction and classification. 3.1. Signal pre-processing Two algorithms are proposed which eliminate the impact of sensor displacement and calibrate the signal dynamically. Further detail on these algorithms can be found in our previous work [14]. 3.2. Sampling frequency Sampling frequency is an important parameter for activity recognition. Maintaining a high sampling rate for the accelerometer-based systems can have a consequent increase in accurately inferring body movements by respecting the NyquistShannon Sampling theorem [15]. However, it also increases the computing load and power requirements. In this section, we present the recognition accuracies versus the sampling rate increase, using four previously well validated classifiers. The mean and variance features were adopted, and a non-over-lapping window size of 1s was selected based on [16]. The experiment was executed using the following sensor combinations: thigh, chest, waist, side of body and multi-sensor (four sensors together). The overall recognition accuracy of the individual activities listed in Table 2 was obtained with the 10-fold cross-validation method. The experiment was repeated by increasing the sampling rate from 10 to 200 Hz at 10 Hz increments and the classification accuracies were noted.
calculation was executed 5000 times with different testing samples, and the average computing execution time was obtained. 3.4. Classifier selection There has been a large amount of published work recently on the applications of various types of classifiers for activity recognition [19,20]. The selection of classifiers for activity recognition is determined by a number of factors. In addition to accuracy, the factors such as the ease of development and the speed of real-time execution influence classifier selection. In this section, we compare the recognition accuracy and the execution time of typical classifiers using exactly the same validation setting, which gives an insight into the performance of these classifiers. Five classifiers (the ANN classifier [19], the Decision Tree classifier [12], the KNN classifier [17], the Naïve Bayes classifier [21] and the SVM classifier [22]) was compared with each other. Four sensors were used to recognize five activities as shown in Table 2. A sampling rate of 20 Hz was used, and a non-overlapping window size of 1s were adopted. The mean and var features were extracted from the acceleration data, and these features were then used in the classifiers. A 10-fold cross-validation was used to evaluate the recognition accuracy of those classifiers. The comparison of these classifiers was based on the following measurements: the recognition accuracy, the training time and the testing time. The training and testing times are defined as the length of time a classifier spends on building and testing the model respectively.
Table 3 Features and applications. Category
Feature
Reference
Time-domain
Mean Standard derivation and variance Zero or mean crossing rate Root mean square Peak count
[28,29] [21] [18] [18,29] [30]
Frequency-domain
Spectral energy Spectral entropy Spectral centroid
[12] [17] [31]
Heuristic and others
Signal magnitude area Inter-axis correlation coefficient Tilt angle Angle velocity
[29,32] [12] [19] [33]
3.3. Feature selection Table 3 presents the most widely used features and their applications. A number of studies have already compared the recognition accuracy of these features [12,17]. Hence, we focus on investigating the execution time of computing these features. The authors in [18] presented a pilot study of the times required to compute the relevant features, by determining the average CPU cycles and computation time required on a sensor platform. In this study, the calculation of these features was implemented on a laptop computer (Intel Core 2 Duo CPU 2.40GHz and 2GB RAM). Each
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model
ARTICLE IN PRESS
JJBE-2429; No. of Pages 7
L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
4
Table 4 Different wearable systems with a single sensor or multiple sensors. System name
Description (sensor, sampling rate, window size, features, classifier
Single-Sensor A Single-Sensor B
Chest, 50 Hz, 0.5 s, (mean, var), Naïve Bayes. Chest, 50 Hz, 0.5 s, (mean, var, sma, ta, coeff), ANN Waist, 50 Hz, 0.5 s, (mean, energy, entropy), Decision Tree (Chest, waist, thigh, side), 20 Hz, 1 s, (mean, var, energy), Naïve Bayes (Chest, waist, thigh, side), 20 Hz, 1 s, (mean, var), Decision Tree (Chest, waist, thigh, side), 20 Hz, 1 s, (mean), Decision Tree
Single-Sensor C Multi-Sensor A Multi-Sensor B Multi-Sensor C
re-implement these studies, we simplified these systems as shown in Table 4. The Single-Sensor A system is defined and simplified based upon [23], the Single-Sensor B system is based upon [19], and the Single-Sensor C system upon [12]. 4. Results Investigations were undertaken on the impact of sampling frequency, the calculation effort required for various features, and the performance of various classifiers. The results of these studies are presented next, followed by the performance analysis of the different sensor systems. 4.1. Sampling frequency
3.5. Multi-sensor vs. single-sensor In this section, six representative wearable systems with single sensor or multiple sensors are compared. These systems are shown in Table 4 and can be partitioned into two categories: the single sensor system (Single-Sensor A, B and C) with the more intensive computing algorithms and the multi-sensor systems (Multi-Sensor A, B and C) with light-weight tasks running on each sensor. For the multi-sensor systems, the Naïve Bayes classifier and the Decision Trees classifier were used as the classification algorithms. A fixed window size of 1s and a sampling rate of 20 Hz were adopted for these systems as previously described. For the Naïve Bayes classifier, the mean, var and spectral energy were adopted as the features due to its weak recognition ability. Two feature combinations were respectively used for the Decision Tree classifier: the mean only and the combination of mean and var. For the single sensor systems, a fixed window size of 0.5s and a sampling frequency of 50 Hz were adopted, since the single sensor systems were more sensitive to sampling frequencies below 50 Hz as illustrated in Fig. 3. The classifiers and the features were chosen based on published studies. As the objective of this study was not to
The experimental results for comparing the sampling frequency versus the classification accuracy are shown in Fig. 3. Fig. 3 illustrates that the recognition accuracy variations versus the increase of the sampling rate increases marginally by just 1% above 20 Hz and stabilizes beyond 50 Hz. Fig. 3 also illustrates that the wearable system adopting multiple sensors is less sensitive to the sampling rate than those using a single sensor. Although the high sampling rate can increase the recognition accuracy, it also leads to a several fold increase in computing load [13]. Thus, a sampling frequency of 20 Hz is proposed as the sampling frequency for the wearable system using multiple sensors. 4.2. Features Fig. 4 shows the average execution time to extract the features. According to the execution time, these features can be categorized into three levels: (I) the Mean (mean), zero crossing rate (zcr), mean crossing rate (mcr), root mean square (rms), spectral energy (energy) and signal magnitude area (sma) features, whose execution time is approximately 20 s in our experimental environment. (II) The standard deviation (std) and variance (var) features, whose
Fig. 3. The recognition accuracies versus the increase of the sampling rate using different classifiers. Thigh (the single thigh sensor), Chest (the single chest sensor), side (the single side of body sensor), waist (the single waist sensor) and multi (four sensors together).
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model
ARTICLE IN PRESS
JJBE-2429; No. of Pages 7
L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
5
Fig. 4. The average time of extracting popular features: Mean (mean), standard deviation (std), variance (var), zero crossing rate (zcr), mean crossing rate (mcr), root mean square (rms), inter-axis correlation coefficient (coeff), spectral energy (energy), spectral entropy (entropy), signal magnitude area (sma), tilt angle (ta) and angle velocity ( ).
execution time is around 50 s. (III) The inter-axis correlation coefficient (coeff), spectral entropy (entropy), tilt angle (ta) and angle velocity ( ) features, whose execution time is greater than 100 s. Considering the recognition accuracy obtained from previous studies [17] and the execution time, the mean and var features were proposed for the multi-sensor system. Although these features are sensitive to changes in sensor orientation, the algorithm described in the signal pre-processing section compensates for this problem. The scatter plot of all subjects with the sensors attached to thigh and chest is presented in Fig. 5. As the accelerometer signal was transformed into a horizontal–vertical coordinate system detailed in [14], the activities were scattered in a 2D plot.
4.3. Classifiers The experimental results are shown in Table 5. Table 5 shows that the ANN, Decision Trees and KNN classifiers achieve a recognition accuracy of ≥96%, which is greater than the Naïve Bayes classifier and the SVM classifier. However, long training and testing times are required for both the ANN and KNN classifiers. The Decision Tree classifier achieved the second highest recognition accuracy in this experiment. In addition, the combination of training time and testing time is the second lowest among the classifiers. Thus, the Decision Tree classifier is identified as an efficient classifier for the wearable system using multiple sensors.
4.4. Multi-sensor vs. single-sensor The results comparing the multi-sensor systems with the singlesensor systems are shown in Table 6. Table 6 illustrates that the Multi-Sensor B system obtains the highest recognition accuracy of 96.4% despite employing simple features (only mean and var) and a lightweight classifier (the Decision Tree classifier). The Multi-Sensor C system is similar to the Multi-Sensor B system except it only implements the mean feature. Table 6 shows the recognition accuracy only decreases by 1.9% while removing the var feature from the Multi-Sensor B system. The Single-Sensor B and SingleSensor C systems achieve a recognition accuracy of >90%, and the Single-Sensor C system achieves the highest recognition accuracy of 92.8% in the single-sensor category, which is 3.6% lower than the Multi-Sensor B system. The Multi-Sensor A system and the Single-Sensor A system achieve the lowest two recognition accuracies and both adopted the Naïve Bayes classifier. Table 7 shows the confusion matrices of the Multi-Sensor B system and the Single-Sensor C system, which obtain the highest recognition accuracy in the two categories, respectively. The Multi-Sensor B system obtains the average recognition accuracies of 97.9% for Static Activities and 94.0% for Dynamic Activities and Transitional Activities, but the Single-Sensor C system achieves the average recognition accuracies of 97.0% for Static Activities and 85.6% for Dynamic Activities and Transitional Activities.
Table 6 The comparison of recognition accuracies of the ambulatory systems.
Table 5 The comparison of the various classifiers. Classifiers
Recognition accuracy (%)
Training time (s)
Testing time (s)
ANN Decision tree KNN Naïve Bayes SVM
96.8 96.4 96.2 89.5 92.7
490.85 3.55 0.05 0.31 61.69
0.81 0.36 56.52 1.19 0.44
System name
Recognition accuracy (%)
Multi-Sensor B Multi-Sensor C Single-Sensor C Single-Sensor B Single-Sensor A Multi-Sensor A
96.4 94.5 92.8 90.9 81.9 83.2
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model
ARTICLE IN PRESS
JJBE-2429; No. of Pages 7
L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
6
Fig. 5. The scatter plot of all subjects with the sensors attached to thigh and chest. The top left is the scatter plot of mean from thigh. The top right is the scatter plot of var from thigh. The bottom left is the scatter plot of mean from chest. The bottom right is the scatter plot of var from chest.
Table 7 The confusion matrix of the multi-sensor B system and the single-sensor C system. Standing
Sitting
Lying
Walking
Transition
Multi-Sensor B
Standing Sitting Lying Walking Transition
6223 3 0 56 46
0 5742 2 1 65
0 0 683 0 9
97 2 0 5104 191
25 138 6 86 1965
Single-Sensor C
Standing Sitting Lying Walking Transition
6149 18 0 81 84
18 5709 0 12 86
0 0 670 0 19
130 8 0 4855 507
46 149 19 294 1576
5. Discussion and conclusion In this study, the design of the multi-sensor system consisted of signal pre-processing algorithms, sampling frequency selection, feature selection and classifier selection. In the signal pre-processing stage, two algorithms were proposed to make the collected dataset insensitive to changes in the experimental environment. The impact of the sampling rate on the recognition accuracy was then investigated using four classifiers. The experimental results illustrated that the recognition accuracy was steady at 50 Hz and above, and the single sensor system was more sensitive to the sampling rate than the multi-sensor system. Considering the trade-off between recognition accuracy, wireless communication overhead and the computing load, we adopted the sampling rate of 20 Hz for the multi-sensor systems. For feature selection, most of the previous studies have deeply investigated the recognition accuracy performance of specific features. The execution time for extracting features was presented. As the multi-sensor
system could obtain the mobility information from different positions, we implemented two simple time-domain features for each sensor: mean and var. A further study on the selection of specific feature combinations, such as that performed [9], is worth performing in-order to investigate if additional computationally un-intensive features can be added. Five classifiers were compared using the same settings. The comparison measurements involved the recognition accuracy, the training time and the testing time. The experimental results illustrated that the Decision Tree classifier was most suitable for the multi-sensor system, and it obtained the second highest recognition accuracy with the second lowest execution time. The ANN, KNN and SVM classifiers have been investigated and have shown good performance accuracy. However, this work showed that these three classifiers were computationally intensive in the training or testing stages. In the comparison presented, six wearable systems employing a single sensor or multiple sensors were compared using the same dataset. This comparison validates the proposed method:
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model JJBE-2429; No. of Pages 7
ARTICLE IN PRESS L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
adopting multiple sensors with light-weight algorithms to obtain higher recognition accuracy. The results show that the multi-sensor system employing just the mean and var features and the Decision Tree classifier obtained the highest recognition accuracy. The single-sensor systems did not surpass the proposed multi-sensor system on the recognition accuracy despite employing a higher sampling rate, more complicated features and a more sophisticated classifier. In addition, the experimental results illustrates that the adoption advantage of the multi-sensor system, compared to the single-sensor system, is more appropriate when recognizing Dynamic Activities and Transitional Activities. It is worth noting that the primary focus of this work is to achieve high recognition accuracy with light-weight algorithms running in each sensor node. Although previous studies have proposed recognizing ADLs using the single sensor based systems, a trade-off between recognition accuracy and computational load is rarely studied. For example, the authors in [19] proposed a single-sensor system adopting augmented-signal features and a hierarchical ANN classifier with a recognition accuracy of 97.9%. However, the features and the classifier adopted are bothcomputationally intensive. Furthermore, using additional ambient sensors also leads to higher recognition accuracy, as proposed in [24,25]. However, these monitoring systems are usually restricted by the applicable environment of the ambient sensors. Most of sensors are integrated into a garment, which can be easily worn by subjects. Within the experiment which included long-term monitoring, none of the subjects complained of any inconvenience due to this multi-sensor system. The end-point for such an activity recognition system is integration into a telemonitoring system, such as that developed during the eCAALYX project [26] which aimed at promoting safer independent living in the older adult population. In this study, the 10-fold cross-validation is adopted to evaluate classifiers. A Monte Carlo method [27] with random sampling is another alternative. The Monte Carlo method can cover the whole data set in both training and testing stage. A comparison between these two methods is warranted. Acknowledgement The authors also wish to acknowledge the eCAALYX project [26] for supporting the datasets. Competing interests: None declared. Funding: Embark Postgraduate Scholarship Scheme from Irish Research Council for Science, Engineering & Technology (IRCSET). Enhanced Complete Ambient Assisted Living Experiment (eCAALYX) funded by the European Commission under the AAL Joint programme. Ethical approval: The Research Ethics Committee of the Faculty of Science and Engineering Ethical at University of Limerick approved the trial protocol. The reference number is S&E10/17. Subjects gave informed consent to participate. References [1] Paillard-Borg S, Wang H-X, Winblad B, Fratiglioni L. Pattern of participation in leisure activities among older people in relation to their health conditions and contextual factors: a survey in a Swedish urban area. Ageing and Society 2009;29:803. [2] Cruz-Jentoft AJ, Baeyens JP, Bauer JM, Boirie Y, Cederholm T, Landi F, et al. Sarcopenia: European consensus on definition and diagnosis: report of the European working group on Sarcopenia in older people. Age and Ageing 2010;39:412–23. [3] Godfrey A, Conway R, Meagher D, OLaighin G. Direct measurement of human movement by accelerometry. Medical Engineering and Physics 2008;30:1364–86. [4] Curone D, Bertolotti G, Cristiani A. A real-time and self-calibrating algorithm based on triaxial accelerometer signals for the detection of human posture and activity. Information 2010;14:1098–105.
7
[5] Parkka J, Ermes M. Activity classification using realistic data from wearable sensors. IEEE Transactions on Information Technology in Biomedicine 2006;10:119–28. [6] Lugade V, Fortune E, Morrow M, Kaufman K. Validity of using tri-axial accelerometers to measure human movement. Part I: posture and movement detection. Medical Engineering and Physics 2013;36(2):169–76. [7] Godfrey A, Bourke AK, Olaighin GM, van de Ven P, Nelson J. Activity classification using a single chest mounted tri-axial accelerometer. Medical Engineering and Physics 2011;33:1127–35. [8] Najafi B, Aminian K, Paraschiv-Ionescu A, Loew F, Büla CJ, Robert P. Ambulatory system for human motion analysis using a kinematic sensor: monitoring of daily physical activity in the elderly. IEEE Transactions on Biomedical Engineering 2003;50:711–23. [9] Palmerini L, Rocchi L, Mellone S, Valzania F, Chiari L. Feature selection for accelerometer-based posture analysis in Parkinson’s disease. IEEE Transactions on Information Technology in Biomedicine 2011;15:481–90. [10] Kwapisz J, Weiss G. Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations 2011;12:74–82. [11] Guiry J, Van de Ven P, Nelson J. Orientation independent human mobility monitoring with an android smartphone. In: Proceedings of the 9th IASTED international conference on biomedical engineering. 2012. [12] Bao L. Activity recognition from user-annotated acceleration data. Pervasive Computing 2004;3001:1–17. [13] Burns A, Greene BR, McGrath MJ, O’Shea TJ, Kuris B, Ayer SM, et al. SHIMMERTM – a wireless sensor platform for noninvasive biomedical research. IEEE Sensors Journal 2010;10:1527–34. [14] Gao L, Bourke AK, Nelson J. A system for activity recognition using multi-sensor fusion. In: Proceedings of 2011 annual international conference of the ieee engineering in medicine and biology society. 2011. p. 7869–72. [15] Marks JR. Introduction to Shannon sampling and interpolation theory. New York: Springer; 1991. [16] Mathie MJ, Coster ACF, Lovell NH, Celler BG. Detection of daily physical activities using a triaxial accelerometer. Medical and Biological Engineering and Computing 2003;41:296–301. [17] Preece SJ, Goulermas JY, Kenney LPJ, Howard D. A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data. IEEE Transactions on Biomedical Engineering 2009;56:871–9. [18] Maurer U, Smailagic A, Siewiorek DP. Activity recognition and monitoring using multiple sensors on different body positions. In: International workshop on wearable and implementable body sensor networks (BSN). 2006. p. 113–6. [19] Khan A, Lee Y, Lee S. A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Transactions on Information Technology in Biomedicine 2010;14:1166–72. [20] Gil GB, Jesús AB, De Lopéz JMM. Using J48 for activity recognition on mobile phones. Advances in Intelligent and Soft Computing 2012;151:141–50. [21] Kern N, Schiele B, Schmidt A. Multi-sensor activity context detection for wearable computing. Ambient Intelligence 2003;2875:220–32. [22] Doukas C. Advanced patient or elder fall detection based on movement and sound data. In: Proceedings of 2nd international conference on pervasive computing technologies for healthcare. 2008. p. 103–7. [23] Yang X, Dinh A, Chen L. Implementation of a wearerable real-time system for physical activity recognition based on naive Bayes classifier. In: 2010 international conference on bioinformatics and biomedical technology (ICBBT). 2010. p. 101–5. [24] Atallah L, Lo B, Ali R, King R, Yang G-Z. Real-time activity classification using ambient and wearable sensors. IEEE Transactions on Information Technology in Biomedicine 2009;13:1031–9. [25] Kasteren T, Van Noulas A, Englebienne G, Krose B. Accurate activity recognition in a home setting. In: Proc. UbiComp 2008. 2008. p. 1–9. [26] Boulos MK, Lou RRC, Kamel Boulos MN, Anastasiou A, Nugent CD, Alexandersson J, et al. Connectivity for healthcare and well-being management: examples from six European projects. International Journal of Environmental Research and Public Health 2009;6:1947–71. [27] Bolstad WM. Introduction to Bayesian statistics. 2nd ed. New Jersey: John Wiley; 2007. [28] Lombriser C, Bharatula NB, Roggen D. On-body activity recognition in a dynamic sensor network. In: Proceedings of the ICST 2nd international conference on body area networks. 2007. p. 1–6. [29] Yang J-Y, Wang J-S, Chen Y-P. Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers. Pattern Recognition Letters 2008;29:2213–20. [30] Ward JA, Lukowicz P, Tröster G. Gesture spotting using wrist worn microphone and 3-axis accelerometer. In: Proceedings of the 2005 joint conference on Smart objects and ambient intelligence innovative context-aware services: usages and technologies. New York, USA: ACM Press; 2005. p. 99. [31] Wu WH, Bui AaT, Batalin Ma, Au LK, Binney JD, Kaiser WJ. MEDIC: medical embedded device for individualized care. Artificial Intelligence in Medicine 2008;42:137–52. [32] Karantonis D, Narayanan M, Mathie M, Lovell NH, Celler BG. Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring. IEEE Transactions on Information Technology in Biomedicine 2006;10:156–67. [33] Bourke AK, van de Ven P, Gamble M, O’Connor R, Murphy K, Bogan E, et al. Evaluation of waist-mounted tri-axial accelerometer based fall-detection algorithms during scripted and continuous unscripted activities. Journal of Biomechanics 2010;43:3051–7.
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
ARTICLE IN PRESS Medical Engineering & Physics xxx (2014) xxx–xxx
Contents lists available at ScienceDirect
Medical Engineering & Physics journal homepage: www.elsevier.com/locate/medengphy
Technical note
Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems Lei Gao ∗ , A.K. Bourke, John Nelson Department of Electronic and Computer Engineering, University of Limerick, Ireland
a r t i c l e
i n f o
Article history: Received 17 August 2012 Received in revised form 8 January 2014 Accepted 8 February 2014 Keywords: Activity recognition Multi-sensor fusion Accelerometer Activities of daily living
a b s t r a c t Physical activity has a positive impact on people’s well-being and it had been shown to decrease the occurrence of chronic diseases in the older adult population. To date, a substantial amount of research studies exist, which focus on activity recognition using inertial sensors. Many of these studies adopt a single sensor approach and focus on proposing novel features combined with complex classifiers to improve the overall recognition accuracy. In addition, the implementation of the advanced feature extraction algorithms and the complex classifiers exceed the computing ability of most current wearable sensor platforms. This paper proposes a method to adopt multiple sensors on distributed body locations to overcome this problem. The objective of the proposed system is to achieve higher recognition accuracy with “light-weight” signal processing algorithms, which run on a distributed computing based sensor system comprised of computationally efficient nodes. For analysing and evaluating the multi-sensor system, eight subjects were recruited to perform eight normal scripted activities in different life scenarios, each repeated three times. Thus a total of 192 activities were recorded resulting in 864 separate annotated activity states. The methods for designing such a multi-sensor system required consideration of the following: signal pre-processing algorithms, sampling rate, feature selection and classifier selection. Each has been investigated and the most appropriate approach is selected to achieve a trade-off between recognition accuracy and computing execution time. A comparison of six different systems, which employ single or multiple sensors, is presented. The experimental results illustrate that the proposed multi-sensor system can achieve an overall recognition accuracy of 96.4% by adopting the mean and variance features, using the Decision Tree classifier. The results demonstrate that elaborate classifiers and feature sets are not required to achieve high recognition accuracies on a multi-sensor system. © 2014 IPEM. Published by Elsevier Ltd. All rights reserved.
1. Introduction The undertaking of moderate to vigorous periods of activities of daily living (ADL) is now widely accepted to promote a healthier lifestyle in the older adult (>65 years) population [1,2]. Recently a significant amount of research has been performed into the application of wearable sensor-based systems for the measurement of the quantity and quality of the physical activities performed. Inertial sensors such as MEMS accelerometers and rate gyroscopes are now widely used for this application [3–5]. There has been significant work on activity recognition using a single sensor attached to different body locations [6–9]. In addition, recent advances in embedded sensor technology in smart phones,
∗ Corresponding author at: Department of Electronic and Computer Engineering, University of Limerick, Limerick, Ireland. Tel.: +353 87 7862690. E-mail address: [email protected] (L. Gao).
combined with their prevalence in modern day society, have promoted their application for the analysis of human movement activity classification [10,11]. However, three challenges primarily exist in these studies. • The recognition accuracy of the implementations on current single sensor platforms has not proven sufficiently accurate for ADL monitoring [12]. • Mobile phones by their nature are not fixed wearable sensors and inevitably periods of activity may be missed when characteristic functions are used (e.g. making a call). The carrying location of phones is often affected by the carrier’s gender (e.g. women may carry their phone in a hand-bag) and therefore the location also needs to be considered when designing the classification algorithm. • The algorithms for the long-term activity recognition with high recognition accuracy are limited by the trade-off between computational load requirements and battery life.
http://dx.doi.org/10.1016/j.medengphy.2014.02.012 1350-4533/© 2014 IPEM. Published by Elsevier Ltd. All rights reserved.
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model JJBE-2429; No. of Pages 7
ARTICLE IN PRESS L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
2
Table 1 Scenario activities. Description Case 1 Case 2 Case 3 Case 4 Case 5 Case 6 Case 7 Case 8
Fig. 1. The data collection system with a garment involving four accelerometerbased sensors across the human body.
This paper thus proposes to use multiple sensors, executing computationally light-weight algorithms, on distributed body locations for more accurate and potentially more specific-activity recognition. The proposed method adopts the distributed computing concept, which will implement computationally inexpensive tasks on each sensor with the potential to achieve a higher degree of recognition accuracy. Thus the main contributions of this work can be summarized as follows: 1. The establishment of a case for multi-sensor systems through experimental observations: Based on the experimental results and previous studies, a wearable system using multiple sensors on different body locations is proposed, that considers both recognition accuracy and computing execution time as the relevant factors. 2. The comparison of wearable systems consisting of a single sensor or multiple sensors: The recognition accuracies of six wearable systems are compared under the same conditions. The objective is to compare two distinct types of wearable systems: single-sensor wearable systems adopting complex algorithms and multi-sensor systems employing light-weight algorithms. 2. Data collection Eight community dwelling older adults ranging in age from 70 to 83 (76.50 ± 4.41 years), capable of walking unaided and having various conditions such as osteoporosis, COPD, leg ulcer and knee replacement were recruited. Subjects were each fitted with four accelerometer based sensors, measuring accelerations at the chest, left under-arm, waist and thigh, as illustrated in Fig. 1. Subjects were fitted with a garment, to which two sensors were attached at the chest (sternum) and left under-arm. A sensor was attached at the waist (right anterior iliac crest of the pelvis) using a custom carry-case and waist-belt. Subjects gave informed consent to participate. The fourth sensor was worn on the thigh (in a pocket), contained within a random oriented padded box (98 mm × 42 mm × 27 mm). This arrangement simulated the shape and size of a smart phone in a protective case. The ShimmerTM wireless sensor platform [13] was used to record the raw triaxial accelerometer data. ShimmerTM is a small sensor platform (53 mm × 32 mm × 25 mm and 22 g) well suited for wearable applications. It includes an on-board tri-axial accelerometer, with a
Sitting down and standing up from an arm chair Sitting down and standing up from a kitchen chair Sitting down and standing up from a toilet seat Walking up and down stairs Sitting down and standing up from a bed Lying down and getting up from a bed Getting in and out of a car seat Walking 10 m
configurable sampling rate up to 1 kHz and an amplitude range up to ±6G. Each subject was asked to perform eight scenario activities, as listed in Table 1, each repeated three times. These were recorded in the subjects’ own home environment. Thus a total of 24 scenarios were recorded from each subject. The sitting-down and standingup activities were recorded in five different life scenarios, so the data set could be used to evaluate the performance of the wearable system in this representative real-life environment. The tri-axial accelerometer data was sampled at 200 Hz and at a resolution of 12 bits, from each of the four sensors and simultaneously transmitted to a laptop computer via a Bluetooth wireless body area network (WBAN). The harvested dataset was automatically annotated during acquisition on the computer. Each scenario commenced and concluded with the subject in a standing position for 5 s, thus 2 transitions were recorded in each scenario (standing-transitionactivity and activity-transition-standing) except for the walking scenarios. The walking scenarios were recorded according to the pattern (standing-walking-standing). These signals were then manually segmented into separate activity states, which belong to the categories listed in Table 2 during post-trial data processing and analysis. There are three categories for the target states: static activities, dynamic activities and transitional activities. Static activities, such as standing, are where a posture is held for a period of time. Dynamic activities involve physical movement, such as walking. Transitional activities were restricted to changing between static activities. In this study, the requirement is to recognize the five activity states listed in Table 2: standing, sitting, lying, walking and transition. The transition activities were grouped and not separately distinguished. The walking activity included both over ground walking and up and down stairs, which were not distinguished. Thus the six sitting and lying scenarios contained five states and the two walking scenarios contained three states. Each scenario was repeated three times, thereby contributing to a total of 108 states per subject. An example of the signal segmentation and annotation can be seen in Fig. 2.
Table 2 Separate activity states. State
Activity
Static activities
Lying Sitting Standing
Dynamic activities
Walking (flat walking and up & down stairs)
Transitional activities
Lying–Standing Standing–Lying Sitting–Standing Standing–Sitting
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model JJBE-2429; No. of Pages 7
ARTICLE IN PRESS L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
3
Fig. 2. The process of dividing and annotating a stand-sit-stand scenario is shown. It is (A) standing, (B) transition, (C) sitting, (D) transition, and (E) standing (1G = 9.81 m/s2 ).
3. Methods In this section, the design of a multi-sensor system is proposed and analyzed. The main steps can be categorized as signal preprocessing, feature extraction and classification. 3.1. Signal pre-processing Two algorithms are proposed which eliminate the impact of sensor displacement and calibrate the signal dynamically. Further detail on these algorithms can be found in our previous work [14]. 3.2. Sampling frequency Sampling frequency is an important parameter for activity recognition. Maintaining a high sampling rate for the accelerometer-based systems can have a consequent increase in accurately inferring body movements by respecting the NyquistShannon Sampling theorem [15]. However, it also increases the computing load and power requirements. In this section, we present the recognition accuracies versus the sampling rate increase, using four previously well validated classifiers. The mean and variance features were adopted, and a non-over-lapping window size of 1s was selected based on [16]. The experiment was executed using the following sensor combinations: thigh, chest, waist, side of body and multi-sensor (four sensors together). The overall recognition accuracy of the individual activities listed in Table 2 was obtained with the 10-fold cross-validation method. The experiment was repeated by increasing the sampling rate from 10 to 200 Hz at 10 Hz increments and the classification accuracies were noted.
calculation was executed 5000 times with different testing samples, and the average computing execution time was obtained. 3.4. Classifier selection There has been a large amount of published work recently on the applications of various types of classifiers for activity recognition [19,20]. The selection of classifiers for activity recognition is determined by a number of factors. In addition to accuracy, the factors such as the ease of development and the speed of real-time execution influence classifier selection. In this section, we compare the recognition accuracy and the execution time of typical classifiers using exactly the same validation setting, which gives an insight into the performance of these classifiers. Five classifiers (the ANN classifier [19], the Decision Tree classifier [12], the KNN classifier [17], the Naïve Bayes classifier [21] and the SVM classifier [22]) was compared with each other. Four sensors were used to recognize five activities as shown in Table 2. A sampling rate of 20 Hz was used, and a non-overlapping window size of 1s were adopted. The mean and var features were extracted from the acceleration data, and these features were then used in the classifiers. A 10-fold cross-validation was used to evaluate the recognition accuracy of those classifiers. The comparison of these classifiers was based on the following measurements: the recognition accuracy, the training time and the testing time. The training and testing times are defined as the length of time a classifier spends on building and testing the model respectively.
Table 3 Features and applications. Category
Feature
Reference
Time-domain
Mean Standard derivation and variance Zero or mean crossing rate Root mean square Peak count
[28,29] [21] [18] [18,29] [30]
Frequency-domain
Spectral energy Spectral entropy Spectral centroid
[12] [17] [31]
Heuristic and others
Signal magnitude area Inter-axis correlation coefficient Tilt angle Angle velocity
[29,32] [12] [19] [33]
3.3. Feature selection Table 3 presents the most widely used features and their applications. A number of studies have already compared the recognition accuracy of these features [12,17]. Hence, we focus on investigating the execution time of computing these features. The authors in [18] presented a pilot study of the times required to compute the relevant features, by determining the average CPU cycles and computation time required on a sensor platform. In this study, the calculation of these features was implemented on a laptop computer (Intel Core 2 Duo CPU 2.40GHz and 2GB RAM). Each
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model
ARTICLE IN PRESS
JJBE-2429; No. of Pages 7
L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
4
Table 4 Different wearable systems with a single sensor or multiple sensors. System name
Description (sensor, sampling rate, window size, features, classifier
Single-Sensor A Single-Sensor B
Chest, 50 Hz, 0.5 s, (mean, var), Naïve Bayes. Chest, 50 Hz, 0.5 s, (mean, var, sma, ta, coeff), ANN Waist, 50 Hz, 0.5 s, (mean, energy, entropy), Decision Tree (Chest, waist, thigh, side), 20 Hz, 1 s, (mean, var, energy), Naïve Bayes (Chest, waist, thigh, side), 20 Hz, 1 s, (mean, var), Decision Tree (Chest, waist, thigh, side), 20 Hz, 1 s, (mean), Decision Tree
Single-Sensor C Multi-Sensor A Multi-Sensor B Multi-Sensor C
re-implement these studies, we simplified these systems as shown in Table 4. The Single-Sensor A system is defined and simplified based upon [23], the Single-Sensor B system is based upon [19], and the Single-Sensor C system upon [12]. 4. Results Investigations were undertaken on the impact of sampling frequency, the calculation effort required for various features, and the performance of various classifiers. The results of these studies are presented next, followed by the performance analysis of the different sensor systems. 4.1. Sampling frequency
3.5. Multi-sensor vs. single-sensor In this section, six representative wearable systems with single sensor or multiple sensors are compared. These systems are shown in Table 4 and can be partitioned into two categories: the single sensor system (Single-Sensor A, B and C) with the more intensive computing algorithms and the multi-sensor systems (Multi-Sensor A, B and C) with light-weight tasks running on each sensor. For the multi-sensor systems, the Naïve Bayes classifier and the Decision Trees classifier were used as the classification algorithms. A fixed window size of 1s and a sampling rate of 20 Hz were adopted for these systems as previously described. For the Naïve Bayes classifier, the mean, var and spectral energy were adopted as the features due to its weak recognition ability. Two feature combinations were respectively used for the Decision Tree classifier: the mean only and the combination of mean and var. For the single sensor systems, a fixed window size of 0.5s and a sampling frequency of 50 Hz were adopted, since the single sensor systems were more sensitive to sampling frequencies below 50 Hz as illustrated in Fig. 3. The classifiers and the features were chosen based on published studies. As the objective of this study was not to
The experimental results for comparing the sampling frequency versus the classification accuracy are shown in Fig. 3. Fig. 3 illustrates that the recognition accuracy variations versus the increase of the sampling rate increases marginally by just 1% above 20 Hz and stabilizes beyond 50 Hz. Fig. 3 also illustrates that the wearable system adopting multiple sensors is less sensitive to the sampling rate than those using a single sensor. Although the high sampling rate can increase the recognition accuracy, it also leads to a several fold increase in computing load [13]. Thus, a sampling frequency of 20 Hz is proposed as the sampling frequency for the wearable system using multiple sensors. 4.2. Features Fig. 4 shows the average execution time to extract the features. According to the execution time, these features can be categorized into three levels: (I) the Mean (mean), zero crossing rate (zcr), mean crossing rate (mcr), root mean square (rms), spectral energy (energy) and signal magnitude area (sma) features, whose execution time is approximately 20 s in our experimental environment. (II) The standard deviation (std) and variance (var) features, whose
Fig. 3. The recognition accuracies versus the increase of the sampling rate using different classifiers. Thigh (the single thigh sensor), Chest (the single chest sensor), side (the single side of body sensor), waist (the single waist sensor) and multi (four sensors together).
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model
ARTICLE IN PRESS
JJBE-2429; No. of Pages 7
L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
5
Fig. 4. The average time of extracting popular features: Mean (mean), standard deviation (std), variance (var), zero crossing rate (zcr), mean crossing rate (mcr), root mean square (rms), inter-axis correlation coefficient (coeff), spectral energy (energy), spectral entropy (entropy), signal magnitude area (sma), tilt angle (ta) and angle velocity ( ).
execution time is around 50 s. (III) The inter-axis correlation coefficient (coeff), spectral entropy (entropy), tilt angle (ta) and angle velocity ( ) features, whose execution time is greater than 100 s. Considering the recognition accuracy obtained from previous studies [17] and the execution time, the mean and var features were proposed for the multi-sensor system. Although these features are sensitive to changes in sensor orientation, the algorithm described in the signal pre-processing section compensates for this problem. The scatter plot of all subjects with the sensors attached to thigh and chest is presented in Fig. 5. As the accelerometer signal was transformed into a horizontal–vertical coordinate system detailed in [14], the activities were scattered in a 2D plot.
4.3. Classifiers The experimental results are shown in Table 5. Table 5 shows that the ANN, Decision Trees and KNN classifiers achieve a recognition accuracy of ≥96%, which is greater than the Naïve Bayes classifier and the SVM classifier. However, long training and testing times are required for both the ANN and KNN classifiers. The Decision Tree classifier achieved the second highest recognition accuracy in this experiment. In addition, the combination of training time and testing time is the second lowest among the classifiers. Thus, the Decision Tree classifier is identified as an efficient classifier for the wearable system using multiple sensors.
4.4. Multi-sensor vs. single-sensor The results comparing the multi-sensor systems with the singlesensor systems are shown in Table 6. Table 6 illustrates that the Multi-Sensor B system obtains the highest recognition accuracy of 96.4% despite employing simple features (only mean and var) and a lightweight classifier (the Decision Tree classifier). The Multi-Sensor C system is similar to the Multi-Sensor B system except it only implements the mean feature. Table 6 shows the recognition accuracy only decreases by 1.9% while removing the var feature from the Multi-Sensor B system. The Single-Sensor B and SingleSensor C systems achieve a recognition accuracy of >90%, and the Single-Sensor C system achieves the highest recognition accuracy of 92.8% in the single-sensor category, which is 3.6% lower than the Multi-Sensor B system. The Multi-Sensor A system and the Single-Sensor A system achieve the lowest two recognition accuracies and both adopted the Naïve Bayes classifier. Table 7 shows the confusion matrices of the Multi-Sensor B system and the Single-Sensor C system, which obtain the highest recognition accuracy in the two categories, respectively. The Multi-Sensor B system obtains the average recognition accuracies of 97.9% for Static Activities and 94.0% for Dynamic Activities and Transitional Activities, but the Single-Sensor C system achieves the average recognition accuracies of 97.0% for Static Activities and 85.6% for Dynamic Activities and Transitional Activities.
Table 6 The comparison of recognition accuracies of the ambulatory systems.
Table 5 The comparison of the various classifiers. Classifiers
Recognition accuracy (%)
Training time (s)
Testing time (s)
ANN Decision tree KNN Naïve Bayes SVM
96.8 96.4 96.2 89.5 92.7
490.85 3.55 0.05 0.31 61.69
0.81 0.36 56.52 1.19 0.44
System name
Recognition accuracy (%)
Multi-Sensor B Multi-Sensor C Single-Sensor C Single-Sensor B Single-Sensor A Multi-Sensor A
96.4 94.5 92.8 90.9 81.9 83.2
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model
ARTICLE IN PRESS
JJBE-2429; No. of Pages 7
L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
6
Fig. 5. The scatter plot of all subjects with the sensors attached to thigh and chest. The top left is the scatter plot of mean from thigh. The top right is the scatter plot of var from thigh. The bottom left is the scatter plot of mean from chest. The bottom right is the scatter plot of var from chest.
Table 7 The confusion matrix of the multi-sensor B system and the single-sensor C system. Standing
Sitting
Lying
Walking
Transition
Multi-Sensor B
Standing Sitting Lying Walking Transition
6223 3 0 56 46
0 5742 2 1 65
0 0 683 0 9
97 2 0 5104 191
25 138 6 86 1965
Single-Sensor C
Standing Sitting Lying Walking Transition
6149 18 0 81 84
18 5709 0 12 86
0 0 670 0 19
130 8 0 4855 507
46 149 19 294 1576
5. Discussion and conclusion In this study, the design of the multi-sensor system consisted of signal pre-processing algorithms, sampling frequency selection, feature selection and classifier selection. In the signal pre-processing stage, two algorithms were proposed to make the collected dataset insensitive to changes in the experimental environment. The impact of the sampling rate on the recognition accuracy was then investigated using four classifiers. The experimental results illustrated that the recognition accuracy was steady at 50 Hz and above, and the single sensor system was more sensitive to the sampling rate than the multi-sensor system. Considering the trade-off between recognition accuracy, wireless communication overhead and the computing load, we adopted the sampling rate of 20 Hz for the multi-sensor systems. For feature selection, most of the previous studies have deeply investigated the recognition accuracy performance of specific features. The execution time for extracting features was presented. As the multi-sensor
system could obtain the mobility information from different positions, we implemented two simple time-domain features for each sensor: mean and var. A further study on the selection of specific feature combinations, such as that performed [9], is worth performing in-order to investigate if additional computationally un-intensive features can be added. Five classifiers were compared using the same settings. The comparison measurements involved the recognition accuracy, the training time and the testing time. The experimental results illustrated that the Decision Tree classifier was most suitable for the multi-sensor system, and it obtained the second highest recognition accuracy with the second lowest execution time. The ANN, KNN and SVM classifiers have been investigated and have shown good performance accuracy. However, this work showed that these three classifiers were computationally intensive in the training or testing stages. In the comparison presented, six wearable systems employing a single sensor or multiple sensors were compared using the same dataset. This comparison validates the proposed method:
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
G Model JJBE-2429; No. of Pages 7
ARTICLE IN PRESS L. Gao et al. / Medical Engineering & Physics xxx (2014) xxx–xxx
adopting multiple sensors with light-weight algorithms to obtain higher recognition accuracy. The results show that the multi-sensor system employing just the mean and var features and the Decision Tree classifier obtained the highest recognition accuracy. The single-sensor systems did not surpass the proposed multi-sensor system on the recognition accuracy despite employing a higher sampling rate, more complicated features and a more sophisticated classifier. In addition, the experimental results illustrates that the adoption advantage of the multi-sensor system, compared to the single-sensor system, is more appropriate when recognizing Dynamic Activities and Transitional Activities. It is worth noting that the primary focus of this work is to achieve high recognition accuracy with light-weight algorithms running in each sensor node. Although previous studies have proposed recognizing ADLs using the single sensor based systems, a trade-off between recognition accuracy and computational load is rarely studied. For example, the authors in [19] proposed a single-sensor system adopting augmented-signal features and a hierarchical ANN classifier with a recognition accuracy of 97.9%. However, the features and the classifier adopted are bothcomputationally intensive. Furthermore, using additional ambient sensors also leads to higher recognition accuracy, as proposed in [24,25]. However, these monitoring systems are usually restricted by the applicable environment of the ambient sensors. Most of sensors are integrated into a garment, which can be easily worn by subjects. Within the experiment which included long-term monitoring, none of the subjects complained of any inconvenience due to this multi-sensor system. The end-point for such an activity recognition system is integration into a telemonitoring system, such as that developed during the eCAALYX project [26] which aimed at promoting safer independent living in the older adult population. In this study, the 10-fold cross-validation is adopted to evaluate classifiers. A Monte Carlo method [27] with random sampling is another alternative. The Monte Carlo method can cover the whole data set in both training and testing stage. A comparison between these two methods is warranted. Acknowledgement The authors also wish to acknowledge the eCAALYX project [26] for supporting the datasets. Competing interests: None declared. Funding: Embark Postgraduate Scholarship Scheme from Irish Research Council for Science, Engineering & Technology (IRCSET). Enhanced Complete Ambient Assisted Living Experiment (eCAALYX) funded by the European Commission under the AAL Joint programme. Ethical approval: The Research Ethics Committee of the Faculty of Science and Engineering Ethical at University of Limerick approved the trial protocol. The reference number is S&E10/17. Subjects gave informed consent to participate. References [1] Paillard-Borg S, Wang H-X, Winblad B, Fratiglioni L. Pattern of participation in leisure activities among older people in relation to their health conditions and contextual factors: a survey in a Swedish urban area. Ageing and Society 2009;29:803. [2] Cruz-Jentoft AJ, Baeyens JP, Bauer JM, Boirie Y, Cederholm T, Landi F, et al. Sarcopenia: European consensus on definition and diagnosis: report of the European working group on Sarcopenia in older people. Age and Ageing 2010;39:412–23. [3] Godfrey A, Conway R, Meagher D, OLaighin G. Direct measurement of human movement by accelerometry. Medical Engineering and Physics 2008;30:1364–86. [4] Curone D, Bertolotti G, Cristiani A. A real-time and self-calibrating algorithm based on triaxial accelerometer signals for the detection of human posture and activity. Information 2010;14:1098–105.
7
[5] Parkka J, Ermes M. Activity classification using realistic data from wearable sensors. IEEE Transactions on Information Technology in Biomedicine 2006;10:119–28. [6] Lugade V, Fortune E, Morrow M, Kaufman K. Validity of using tri-axial accelerometers to measure human movement. Part I: posture and movement detection. Medical Engineering and Physics 2013;36(2):169–76. [7] Godfrey A, Bourke AK, Olaighin GM, van de Ven P, Nelson J. Activity classification using a single chest mounted tri-axial accelerometer. Medical Engineering and Physics 2011;33:1127–35. [8] Najafi B, Aminian K, Paraschiv-Ionescu A, Loew F, Büla CJ, Robert P. Ambulatory system for human motion analysis using a kinematic sensor: monitoring of daily physical activity in the elderly. IEEE Transactions on Biomedical Engineering 2003;50:711–23. [9] Palmerini L, Rocchi L, Mellone S, Valzania F, Chiari L. Feature selection for accelerometer-based posture analysis in Parkinson’s disease. IEEE Transactions on Information Technology in Biomedicine 2011;15:481–90. [10] Kwapisz J, Weiss G. Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations 2011;12:74–82. [11] Guiry J, Van de Ven P, Nelson J. Orientation independent human mobility monitoring with an android smartphone. In: Proceedings of the 9th IASTED international conference on biomedical engineering. 2012. [12] Bao L. Activity recognition from user-annotated acceleration data. Pervasive Computing 2004;3001:1–17. [13] Burns A, Greene BR, McGrath MJ, O’Shea TJ, Kuris B, Ayer SM, et al. SHIMMERTM – a wireless sensor platform for noninvasive biomedical research. IEEE Sensors Journal 2010;10:1527–34. [14] Gao L, Bourke AK, Nelson J. A system for activity recognition using multi-sensor fusion. In: Proceedings of 2011 annual international conference of the ieee engineering in medicine and biology society. 2011. p. 7869–72. [15] Marks JR. Introduction to Shannon sampling and interpolation theory. New York: Springer; 1991. [16] Mathie MJ, Coster ACF, Lovell NH, Celler BG. Detection of daily physical activities using a triaxial accelerometer. Medical and Biological Engineering and Computing 2003;41:296–301. [17] Preece SJ, Goulermas JY, Kenney LPJ, Howard D. A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data. IEEE Transactions on Biomedical Engineering 2009;56:871–9. [18] Maurer U, Smailagic A, Siewiorek DP. Activity recognition and monitoring using multiple sensors on different body positions. In: International workshop on wearable and implementable body sensor networks (BSN). 2006. p. 113–6. [19] Khan A, Lee Y, Lee S. A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Transactions on Information Technology in Biomedicine 2010;14:1166–72. [20] Gil GB, Jesús AB, De Lopéz JMM. Using J48 for activity recognition on mobile phones. Advances in Intelligent and Soft Computing 2012;151:141–50. [21] Kern N, Schiele B, Schmidt A. Multi-sensor activity context detection for wearable computing. Ambient Intelligence 2003;2875:220–32. [22] Doukas C. Advanced patient or elder fall detection based on movement and sound data. In: Proceedings of 2nd international conference on pervasive computing technologies for healthcare. 2008. p. 103–7. [23] Yang X, Dinh A, Chen L. Implementation of a wearerable real-time system for physical activity recognition based on naive Bayes classifier. In: 2010 international conference on bioinformatics and biomedical technology (ICBBT). 2010. p. 101–5. [24] Atallah L, Lo B, Ali R, King R, Yang G-Z. Real-time activity classification using ambient and wearable sensors. IEEE Transactions on Information Technology in Biomedicine 2009;13:1031–9. [25] Kasteren T, Van Noulas A, Englebienne G, Krose B. Accurate activity recognition in a home setting. In: Proc. UbiComp 2008. 2008. p. 1–9. [26] Boulos MK, Lou RRC, Kamel Boulos MN, Anastasiou A, Nugent CD, Alexandersson J, et al. Connectivity for healthcare and well-being management: examples from six European projects. International Journal of Environmental Research and Public Health 2009;6:1947–71. [27] Bolstad WM. Introduction to Bayesian statistics. 2nd ed. New Jersey: John Wiley; 2007. [28] Lombriser C, Bharatula NB, Roggen D. On-body activity recognition in a dynamic sensor network. In: Proceedings of the ICST 2nd international conference on body area networks. 2007. p. 1–6. [29] Yang J-Y, Wang J-S, Chen Y-P. Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers. Pattern Recognition Letters 2008;29:2213–20. [30] Ward JA, Lukowicz P, Tröster G. Gesture spotting using wrist worn microphone and 3-axis accelerometer. In: Proceedings of the 2005 joint conference on Smart objects and ambient intelligence innovative context-aware services: usages and technologies. New York, USA: ACM Press; 2005. p. 99. [31] Wu WH, Bui AaT, Batalin Ma, Au LK, Binney JD, Kaiser WJ. MEDIC: medical embedded device for individualized care. Artificial Intelligence in Medicine 2008;42:137–52. [32] Karantonis D, Narayanan M, Mathie M, Lovell NH, Celler BG. Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring. IEEE Transactions on Information Technology in Biomedicine 2006;10:156–67. [33] Bourke AK, van de Ven P, Gamble M, O’Connor R, Murphy K, Bogan E, et al. Evaluation of waist-mounted tri-axial accelerometer based fall-detection algorithms during scripted and continuous unscripted activities. Journal of Biomechanics 2010;43:3051–7.
Please cite this article in press as: Gao L, et al. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med Eng Phys (2014), http://dx.doi.org/10.1016/j.medengphy.2014.02.012
