Open Access
Issue
Published:
18 June 2014
Activity Recognition with Smartphone Sensors
Ping Ji(
)
)
Cite this article:
Su X, Tong H, Ji P.
Activity Recognition with Smartphone Sensors.
Tsinghua Science and Technology,
2014, 19(3): 235-249.
https://doi.org/10.1109/TST.2014.6838194
Download citation
1046
Views
47
Downloads
Citations
259
Crossref
N/A
WoS
308
Scopus
0
CSCD
The ubiquity of smartphones together with their ever-growing computing, networking, and sensing powers have been changing the landscape of people’s daily life. Among others, activity recoginition, which takes the raw sensor reading as inputs and predicts a user’s motion activity, has become an active research area in recent years. It is the core building block in many high-impact applications, ranging from health and fitness monitoring, personal biometric signature, urban computing, assistive technology, and elder-care, to indoor localization and navigation, etc. This paper presents a comprehensive survey of the recent advances in activity recognition with smartphones’ sensors. We start with the basic concepts such as sensors, activity types, etc. We review the core data mining techniques behind the main stream activity recognition algorithms, analyze their major challenges, and introduce a variety of real applications enabled by activity recognition.
menu
Abstract
Full text
Outline
About this article
Activity Recognition with Smartphone Sensors
Ping Ji(
)
)
Abstract
The ubiquity of smartphones together with their ever-growing computing, networking, and sensing powers have been changing the landscape of people’s daily life. Among others, activity recoginition, which takes the raw sensor reading as inputs and predicts a user’s motion activity, has become an active research area in recent years. It is the core building block in many high-impact applications, ranging from health and fitness monitoring, personal biometric signature, urban computing, assistive technology, and elder-care, to indoor localization and navigation, etc. This paper presents a comprehensive survey of the recent advances in activity recognition with smartphones’ sensors. We start with the basic concepts such as sensors, activity types, etc. We review the core data mining techniques behind the main stream activity recognition algorithms, analyze their major challenges, and introduce a variety of real applications enabled by activity recognition.
Keywords:
machine learning, activity recognition, mobile sensors, data mining, pattern recognition
References(78)
[1]
H. Ye, T. Gu, X. Zhu, J. Xu, X. Tao, J. Lu, and N. Jin, Ftrack: Infrastructure-free floor localization via mobile phone sensing, in Proc. Int. Pervasive Computing and Communications Conf., Lugano, Switzerland, 2012, pp. 2-10.
[2]
A. Ofstad, E. Nicholas, R. Szcodronski, and R. R. Choudhury, Aampl: Accelerometer augmented mobile phone localization, in Proc. 1st Int. Mobile Entity Localization and Tracking in GPS-Less Environments Workshop, San Francisco, USA, 2008, pp. 13-18.
[3]
S. Kozina, H. Gjoreski, M. Gams, and M. Luštrek, Efficient activity recognition and fall detection using accelerometers, in Evaluating AAL Systems Through Competitive Benchmarking, Springer, 2013, pp. 13-23.
[4]
Fitbit, Sensors overview, http://www.fitbit.com/one, 2014, Mar. 17.
[5]
A. Avci, S. Bosch, M. Marin-Perianu, R. Marin-Perianu, and P. Havinga, Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: A survey, in Proc. 23rd Int. Architecture of Computing Systems Conf., Hannover, Germany, 2010, pp. 1-10.
[6]
J. W. Lockhart, T. Pulickal, and G. M. Weiss, Applications of mobile activity recognition, in Proc. 14th Int. Ubiquitous Computing Conf., Seattle, USA, 2012, pp. 1054-1058.
[7]
O. D. Incel, M. Kose, and C. Ersoy, A review and taxonomy of activity recognition on mobile phones, BioNanoScience, vol. 3, no. 2, pp. 145-171, 2013.
[8]
O. D. Lara and M. A. Labrador, A survey on human activity recognition using wearable sensors, Communications Surveys & Tutorials, IEEE, vol. 15, no. 3, pp. 1192-1209, 2013.
[9]
N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell, A survey of mobile phone sensing, Communications Magazine, IEEE, vol. 48, no. 9, pp. 140-150, 2010.
[10]
H. F. Rashvand and K.-F. Hsiao, Smartphone intelligent applications: A brief review, Multimedia Systems, pp. 1-17, 2013.
[11]
H. Becker, M. Borazio, and K. Van Laerhoven, How to log sleeping trends? A case study on the long-term capturing of user data, in Smart Sensing and Context, Springer, 2010, pp. 15-27.
[12]
Google Android, Sensors overview, http://developer.android.com/guide/topics/sensors/sensors_overview.html, 2014, Mar. 01.
[13]
Apple, UIAcceleration class reference, https://developer.apple.com/library/ios/documentation/uikit/reference/UIA- cceleration_Class/Reference/UIAcceleration.html, 2014, Mar. 17.
[14]
K. Muralidharan, A. J. Khan, A. Misra, R. K. Balan, and S. Agarwal, Barometric phone sensors-more hype than hope! http://www.msr-waypoint.com/en-us/people/sagarwal/hotmobile14.pdf, 2014.
[15]
J. Lester, T. Choudhury, and G. Borriello, A practical approach to recognizing physical activities, in Pervasive Computing, Springer, 2006, pp. 1-16.
[16]
L. Bao and S. S. Intille, Activity recognition from user-annotated acceleration data, in Pervasive Computing, Springer, 2004, pp. 1-17.
[17]
Y. E. Ustev, O. Durmaz Incel, and C. Ersoy, User, device and orientation independent human activity recognition on mobile phones: Challenges and a proposal, in Proc. 13th Int. Pervasive and Ubiquitous Computing Adjunct Publication Conf., Zurich, Switzerland, 2013, pp. 1427-1436.
[18]
Y.-S. Lee and S.-B. Cho, Activity recognition using hierarchical hidden markov models on a smartphone with 3d accelerometer, in Hybrid Artificial Intelligent Systems, Springer, 2011, pp. 460-467.
[19]
N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman, Activity recognition from accelerometer data, AAAI, vol. 5, pp. 1541-1546, 2005.
[20]
J. R. Kwapisz, G. M. Weiss, and S. A. Moore, Cell phone-based biometric identification, in Proc. 4th Int. Biometrics: Theory Applications and Systems Conf., Washington DC, USA, 2010, pp. 1-7.
[21]
J. G. Casanova, C. S. Á vila, A. de Santos Sierra, G. B. del Pozo, and V. J. Vera, A real-time in-air signature biometric technique using a mobile device embedding an accelerometer, in Networked Digital Technologies, Springer, 2010, pp. 497-503.
[22]
F. Albinali, M. S. Goodwin, and S. Intille, Detecting stereotypical motor movements in the classroom using accelerometry and pattern recognition algorithms, Pervasive and Mobile Computing, vol. 8, no. 1, pp. 103-114, 2012.
[23]
A. M. Khan, Y.-K. Lee, S. Y. Lee, and T.-S. Kim, A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer, Information Technology in Biomedicine, vol. 14, no. 5, pp. 1166-1172, 2010.
[24]
S. Kaghyan and H. Sarukhanyan, Activity recognition using k-nearest neighbor algorithm on smartphone with tri-axial accelerometer, International Journal of Informatics Models and Analysis (IJIMA), ITHEA International Scientific Society, Bulgaria, vol. 1, pp. 146-156, 2012.
[25]
T. Brezmes, J.-L. Gorricho, and J. Cotrina, Activity recognition from accelerometer data on a mobile phone, in Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living, Springer, 2009, pp. 796-799.
[26]
E. Mitchell and D. Monaghan, Classification of sporting activities using smartphone accelerometers, Sensors, vol. 13, no. 4, pp. 5317-5337, 2013.
[27]
T. Choudhury, S. Consolvo, B. Harrison, J. Hightower, A. LaMarca, L. LeGrand, A. Rahimi, A. Rea, G. Bordello, B. Hemingway, et al., The mobile sensing platform: An embedded activity recognition system, Pervasive Computing, IEEE, vol. 7, no. 2, pp. 32-41, 2008.
[28]
L. Wang, T. Gu, X. Tao, and J. Lu, A hierarchical approach to real-time activity recognition in body sensor networks, Pervasive and Mobile Computing, vol. 8, no. 1, pp. 115-130, 2012.
[29]
C. Seeger, A. Buchmann, and K. Van Laerhoven, Myhealthassistant: A phone-based body sensor network that captures the wearer’s exercises throughout the day, in Proc. 6th Int. Body Area Networks Conf., Beijing, China, 2011, pp. 1-7.
[30]
E. Garcia-Ceja and R. Brena, Long-term activity recognition from accelerometer data, Procedia Technology, vol. 7, pp. 248-256, 2013.
[31]
D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine, in Ambient Assisted Living and Home Care, Springer, 2012, pp. 216-223.
[32]
J. R. Kwapisz, G. M. Weiss, and S. A. Moore, Activity recognition using cell phone accelerometers, ACM SigKDD Explorations Newsletter, vol. 12, no. 2, pp. 74-82, 2011.
[33]
S. Wang, J. Yang, N. Chen, X. Chen, and Q. Zhang, Human activity recognition with user-free accelerometers in the sensor networks, in Proc. 2005 Int. Neural Networks and Brain Conf., 2005, pp. 1212-1217.
[34]
Y. Wang, J. Lin, M. Annavaram, Q. A. Jacobson, J. Hong, B. Krishnamachari, and N. Sadeh, A framework of energy efficient mobile sensing for automatic user state recognition, in Proc. 7th Int. Conference on Mobile Systems, Applications, and Services Conf., Kraków, Poland, 2009, pp. 179-192.
[35]
P. Siirtola and J. Röning, Recognizing human activities user-independently on smartphones based on accelerometer data, International Journal of Interactive Multimedia & Artificial Intelligence, vol. 1, no. 5, pp. 38-45, 2012.
[36]
E. Miluzzo, N. D. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X. Zheng, and A. T. Campbell, Sensing meets mobile social networks: The design, implementation and evaluation of the cenceme application, in Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems, ACM, 2008, pp. 337-350.
[37]
J. S. Carós, O. Chételat, P. Celka, S. Dasen, and J. CmÃral, Very low complexity algorithm for ambulatory activity classification, presented at the 3rd European Medical and Biological Conference, Prague, Czech, 2005.
[38]
S. R. Garner, Weka: The waikato environment for knowledge analysis, in Proc. 2nd New Zealand Computer Science Research Students Conf., Hamilton, New Zealand, 1995, pp. 57-64.
[40]
D. Maguire and R. Frisby, Comparison of feature classification algorithm for activity recognition based on accelerometer and heart rate data, presented at the 9th IT & T Conference, Dubrin, Ireland, 2009.
[41]
H. Martín, A. M. Bernardos, J. Iglesias, and J. R. Casar, Activity logging using lightweight classification techniques in mobile devices, Personal and Ubiquitous Computing, vol. 17, no. 4, pp. 675-695, 2013.
[42]
R. Kohavi, The power of decision tables, in Machine Learning: ECML-95, Springer, 1995, pp. 174-189.
[43]
Wikipedia, k-nearest neighbors algorithm, http://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm, 2014, Mar. 17.
[44]
C. Lombriser, N. B. Bharatula, D. Roggen, and G. Tröster, On-body activity recognition in a dynamic sensor network, in Proc. 2nd Int. Body Area Networks Conf., Florence, Italy, 2007, pp. 17-22.
[45]
M. Kose, O. D. Incel, and C. Ersoy, Online human activity recognition on smart phones, in Workshop on Mobile Sensing: From Smartphones and Wearables to Big Data, Beijing, China, 2012, pp. 11-15.
[46]
P. Zappi, C. Lombriser, T. Stiefmeier, E. Farella, D. Roggen, L. Benini, and G. Tröster, Activity recognition from on-body sensors: Accuracy-power trade-off by dynamic sensor selection, in Wireless Sensor Networks, Springer, 2008, pp. 17-33.
[47]
N. Oliver and E. Horvitz, A comparison of hmms and dynamic bayesian networks for recognizing office activities, in User Modeling, Springer, 2005, pp. 199-209.
[48]
R. K. Ibrahim, E. Ambikairajah, B. G. Celler, and N. H. Lovell, Time-frequency based features for classification of walking patterns, in Proc. 15th Int. Digital Signal Processing Conf., Wales, UK, 2007, pp. 187-190.
[49]
F. R. Allen, E. Ambikairajah, N. H. Lovell, and B. G. Celler, An adapted gaussian mixture model approach to accelerometry-based movement classification using time-domain features, in Proc. 28th Int. Engineering in Medicine and Biology Society Conf., California, USA, 2006, pp. 3600-3603.
[50]
N. Győrbíró, Á. Fábián, and G. Hományi, An activity recognition system for mobile phones, Mobile Networks and Applications, vol. 14, no. 1, pp. 82-91, 2009.
[51]
I. Anderson, J. Maitland, S. Sherwood, L. Barkhuus, M. Chalmers, M. Hall, B. Brown, and H. Muller, Shakra: Tracking and sharing daily activity levels with unaugmented mobile phones, Mobile Networks and Applications, vol. 12, no. 2-3, pp. 185-199, 2007.
[52]
J.-Y. Yang, J.-S. Wang, and Y.-P. Chen, Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers, Pattern Recognition Letters, vol. 29, no. 16, pp. 2213-2220, 2008.
[53]
N. D. Lane, M. Mohammod, M. Lin, X. Yang, H. Lu, S. Ali, A. Doryab, E. Berke, T. Choudhury, and A. Campbell, Bewell: A smartphone application to monitor, model and promote wellbeing, in Proc. 5th Int. ICST Conference on Pervasive Computing Technologies for Healthcare Conf., Dublin, Ireland, 2011, pp. 23-26.
[54]
T. Saponas, J. Lester, J. Froehlich, J. Fogarty, and J. Landay, Ilearn on the iphone: Real-time human activity classification on commodity mobile phones, University of Washington CSE Tech ReportUW-CSE-08-04-02, 2008.
[55]
M. Berchtold, M. Budde, D. Gordon, H. R. Schmidtke, and M. Beigl, Actiserv: Activity recognition service for mobile phones, in Proc. 9th Int. Wearable Computers Conf., Shanghai, China, 2010, pp. 1-8.
[56]
L. Todorovski and S. Džeroski, Combining classifiers with meta decision trees, Machine Learning, 2003, vol. 50, no. 3, pp. 223-249.
[57]
M. Zhang and A. A. Sawchuk, Motion primitive-based human activity recognition using a bag-of-features approach, in Proc. 2nd Int. Health Informatics Symposium, Miami, USA, 2012, pp. 631-640.
[58]
P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, in Proc. 17th Int. Computer Vision and Pattern Recognition Conf., Kauai, USA, 2001.
[59]
A. Reiss, Personalized mobile physical activity monitoring for everyday life, PhD dissertation. Dept. CS. Technical University of Kaiserslautern, Kaiserslautern, German, 2014.
[60]
W.-Y. Deng, Q.-H. Zheng, and Z.-M. Wang, Cross-person activity recognition using reduced kernel extreme learning machine, Neural Networks, 2014.
[61]
J.-G. Park, A. Patel, D. Curtis, S. Teller, and J. Ledlie, Online pose classification and walking speed estimation using handheld devices, in Proc. 14th Int. Ubiquitous Computing Conf., Pittsburgh, USA, 2012, pp. 113-122.
[62]
M. Khan, S. I. Ahamed, M. Rahman, and R. O. Smith, A feature extraction method for realtime human activity recognition on cell phones, in Proc. 3rd Int. Symposium on Quality of Life Technology Conf., Toroto, Canada, 2007.
[63]
Z. Yan, V. Subbaraju, D. Chakraborty, A. Misra, and K. Aberer, Energy-efficient continuous activity recognition on mobile phones: An activity-adaptive approach, in Proc. 16th Int. Wearable Computers Symposium, Newcastle, UK, 2012, pp. 17-24.
[64]
Y. Liang, X. Zhou, Z. Yu, and B. Guo, Energy-efficient motion related activity recognition on mobile devices for pervasive healthcare, Mobile Networks and Applications, pp. 1-15, 2013.
[65]
D. Guan, W. Yuan, Y.-K. Lee, A. Gavrilov, and S. Lee, Activity recognition based on semisupervised learning, in Proc. 13th Int. Embedded and Real-Time Computing Systems and Applications Conf., Daegu, Korea, 2007, pp. 469-475.
[66]
A. Blum and T. Mitchell, Combining labeled and unlabeled data with co-training, in Proc. 11th Int. Computational Learning Theory Conf., Wisconsin, USA, 1998, pp. 92-100.
[67]
M. Mahdaviani and T. Choudhury, Fast and scalable training of semi-supervised crfs with application to activity recognition, in NIPS, 2007, vol. 20, pp. 977-984.
[68]
J. Xie and M. S. Beigi, A scale-invariant local descriptor for event recognition in 1d sensor signals, in Proc. Int. Multimedia and Expo Conf., New York, USA, 2009, pp. 1226-1229.
[69]
D. G. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[70]
A. M. Khan, Human activity recognition using a single tri-axial accelerometer, Ph.D dissertation, Dept. Computer Engineering, Kyung Hee University, Seoul, Korea, 2011.
[71]
Apple: Nike + iPod application, http://www.apple.com/ipod/nike/, 2014.
[72]
Sleepcycle, http://www.sleepcycle.com/, 2014.
[73]
M. Goel, L. Findlater, and J. Wobbrock, Walktype: Using accelerometer data to accomodate situational impairments in mobile touch screen text entry, in Proc. Human Factors in Computing Systems, 2012, pp. 2687-2696.
[74]
P. Marquardt, A. Verma, H. Carter, and P. Traynor, (sp) iphone: Decoding vibrations from nearby keyboards using mobile phone accelerometers, in Proceedings of the 18th ACM Conference on Computer and Communications Security, ACM, 2011, pp. 551-562.
[75]
H. Si, S. J. Kim, N. Kawanishi, and H. Morikawa, A context-aware reminding system for daily activities of dementia patients, in Proc. 27th Int. Distributed Computing Systems Conf., Toroto, Canada, 2007, pp. 50-50.
[76]
W. Song, J. Lee, H. G. Schulzrinne, and B. Lee, Finding 9-1-1 callers in tall buildings, http://academiccommons.columbia.edu/item/ac:156057, 2014.
[77]
L. Mo, S. Liu, R. X. Gao, D. John, J. Staudenmayer, and P. Freedson, Zigbee-based wireless multisensor system for physical activity assessment, in Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, 2011, pp. 846-849.
[78]
Wearit sensors overview, http://www.wearitatwork.com/home/discovering-ubiquity/, 2014.
Publication history
Copyright
Acknowledgements
Rights and permissions
Publication history
Received: 01 April 2014
Revised: 15 April 2014
Accepted: 17 April 2014
Published:
18 June 2014
Issue date: June 2014
Copyright
© The author(s) 2014
Acknowledgements
This material is based upon work supported by the National Science Foundation (Nos. IIS1017415 and CNS-0904901). Research was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-09-2-0053. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Laboratory or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on. Funding was provided by Defense Advanced Research Projects Agency (DARPA) under Contract Number W911NF-11-C-0200 and W911NF-12-C-0028. The content of the information in this document does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation here on.
FEEDBACK 
TOP