AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
PDF (841.6 KB)
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Open Access

Activity Recognition with Smartphone Sensors

Computer Science Department, Graduate Center, the City University of New York, New York, NY 10016, USA.
Computer Science Department, City College, CUNY, New York, NY 10031, USA.
Computer Science Department, Graduate Center, the City University of New York, New York, NY 10016, USA.
Show Author Information

Abstract

The ubiquity of smartphones together with their ever-growing computing, networking, and sensing powers have been changing the landscape of people’s daily life. Among others, activity recoginition, which takes the raw sensor reading as inputs and predicts a user’s motion activity, has become an active research area in recent years. It is the core building block in many high-impact applications, ranging from health and fitness monitoring, personal biometric signature, urban computing, assistive technology, and elder-care, to indoor localization and navigation, etc. This paper presents a comprehensive survey of the recent advances in activity recognition with smartphones’ sensors. We start with the basic concepts such as sensors, activity types, etc. We review the core data mining techniques behind the main stream activity recognition algorithms, analyze their major challenges, and introduce a variety of real applications enabled by activity recognition.

References

[1]
H. Ye, T. Gu, X. Zhu, J. Xu, X. Tao, J. Lu, and N. Jin, Ftrack: Infrastructure-free floor localization via mobile phone sensing, in Proc. Int. Pervasive Computing and Communications Conf., Lugano, Switzerland, 2012, pp. 2-10.
[2]
A. Ofstad, E. Nicholas, R. Szcodronski, and R. R. Choudhury, Aampl: Accelerometer augmented mobile phone localization, in Proc. 1st Int. Mobile Entity Localization and Tracking in GPS-Less Environments Workshop, San Francisco, USA, 2008, pp. 13-18.
[3]
S. Kozina, H. Gjoreski, M. Gams, and M. Luštrek, Efficient activity recognition and fall detection using accelerometers, in Evaluating AAL Systems Through Competitive Benchmarking, Springer, 2013, pp. 13-23.
[4]
Fitbit, Sensors overview, http://www.fitbit.com/one, 2014, Mar. 17.
[5]
A. Avci, S. Bosch, M. Marin-Perianu, R. Marin-Perianu, and P. Havinga, Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: A survey, in Proc. 23rd Int. Architecture of Computing Systems Conf., Hannover, Germany, 2010, pp. 1-10.
[6]
J. W. Lockhart, T. Pulickal, and G. M. Weiss, Applications of mobile activity recognition, in Proc. 14th Int. Ubiquitous Computing Conf., Seattle, USA, 2012, pp. 1054-1058.
[7]
O. D. Incel, M. Kose, and C. Ersoy, A review and taxonomy of activity recognition on mobile phones, BioNanoScience, vol. 3, no. 2, pp. 145-171, 2013.
[8]
O. D. Lara and M. A. Labrador, A survey on human activity recognition using wearable sensors, Communications Surveys & Tutorials, IEEE, vol. 15, no. 3, pp. 1192-1209, 2013.
[9]
N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell, A survey of mobile phone sensing, Communications Magazine, IEEE, vol. 48, no. 9, pp. 140-150, 2010.
[10]
H. F. Rashvand and K.-F. Hsiao, Smartphone intelligent applications: A brief review, Multimedia Systems, pp. 1-17, 2013.
[11]
H. Becker, M. Borazio, and K. Van Laerhoven, How to log sleeping trends? A case study on the long-term capturing of user data, in Smart Sensing and Context, Springer, 2010, pp. 15-27.
[12]
Google Android, Sensors overview, http://developer.android.com/guide/topics/sensors/sensors_overview.html, 2014, Mar. 01.
[14]
K. Muralidharan, A. J. Khan, A. Misra, R. K. Balan, and S. Agarwal, Barometric phone sensors-more hype than hope! http://www.msr-waypoint.com/en-us/people/sagarwal/hotmobile14.pdf, 2014.
[15]
J. Lester, T. Choudhury, and G. Borriello, A practical approach to recognizing physical activities, in Pervasive Computing, Springer, 2006, pp. 1-16.
[16]
L. Bao and S. S. Intille, Activity recognition from user-annotated acceleration data, in Pervasive Computing, Springer, 2004, pp. 1-17.
[17]
Y. E. Ustev, O. Durmaz Incel, and C. Ersoy, User, device and orientation independent human activity recognition on mobile phones: Challenges and a proposal, in Proc. 13th Int. Pervasive and Ubiquitous Computing Adjunct Publication Conf., Zurich, Switzerland, 2013, pp. 1427-1436.
[18]
Y.-S. Lee and S.-B. Cho, Activity recognition using hierarchical hidden markov models on a smartphone with 3d accelerometer, in Hybrid Artificial Intelligent Systems, Springer, 2011, pp. 460-467.
[19]
N. Ravi, N. Dandekar, P. Mysore, and M. L. Littman, Activity recognition from accelerometer data, AAAI, vol. 5, pp. 1541-1546, 2005.
[20]
J. R. Kwapisz, G. M. Weiss, and S. A. Moore, Cell phone-based biometric identification, in Proc. 4th Int. Biometrics: Theory Applications and Systems Conf., Washington DC, USA, 2010, pp. 1-7.
[21]
J. G. Casanova, C. S. Á vila, A. de Santos Sierra, G. B. del Pozo, and V. J. Vera, A real-time in-air signature biometric technique using a mobile device embedding an accelerometer, in Networked Digital Technologies, Springer, 2010, pp. 497-503.
[22]
F. Albinali, M. S. Goodwin, and S. Intille, Detecting stereotypical motor movements in the classroom using accelerometry and pattern recognition algorithms, Pervasive and Mobile Computing, vol. 8, no. 1, pp. 103-114, 2012.
[23]
A. M. Khan, Y.-K. Lee, S. Y. Lee, and T.-S. Kim, A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer, Information Technology in Biomedicine, vol. 14, no. 5, pp. 1166-1172, 2010.
[24]
S. Kaghyan and H. Sarukhanyan, Activity recognition using k-nearest neighbor algorithm on smartphone with tri-axial accelerometer, International Journal of Informatics Models and Analysis (IJIMA), ITHEA International Scientific Society, Bulgaria, vol. 1, pp. 146-156, 2012.
[25]
T. Brezmes, J.-L. Gorricho, and J. Cotrina, Activity recognition from accelerometer data on a mobile phone, in Distributed Computing, Artificial Intelligence, Bioinformatics, Soft Computing, and Ambient Assisted Living, Springer, 2009, pp. 796-799.
[26]
E. Mitchell and D. Monaghan, Classification of sporting activities using smartphone accelerometers, Sensors, vol. 13, no. 4, pp. 5317-5337, 2013.
[27]
T. Choudhury, S. Consolvo, B. Harrison, J. Hightower, A. LaMarca, L. LeGrand, A. Rahimi, A. Rea, G. Bordello, B. Hemingway, et al., The mobile sensing platform: An embedded activity recognition system, Pervasive Computing, IEEE, vol. 7, no. 2, pp. 32-41, 2008.
[28]
L. Wang, T. Gu, X. Tao, and J. Lu, A hierarchical approach to real-time activity recognition in body sensor networks, Pervasive and Mobile Computing, vol. 8, no. 1, pp. 115-130, 2012.
[29]
C. Seeger, A. Buchmann, and K. Van Laerhoven, Myhealthassistant: A phone-based body sensor network that captures the wearer’s exercises throughout the day, in Proc. 6th Int. Body Area Networks Conf., Beijing, China, 2011, pp. 1-7.
[30]
E. Garcia-Ceja and R. Brena, Long-term activity recognition from accelerometer data, Procedia Technology, vol. 7, pp. 248-256, 2013.
[31]
D. Anguita, A. Ghio, L. Oneto, X. Parra, and J. L. Reyes-Ortiz, Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine, in Ambient Assisted Living and Home Care, Springer, 2012, pp. 216-223.
[32]
J. R. Kwapisz, G. M. Weiss, and S. A. Moore, Activity recognition using cell phone accelerometers, ACM SigKDD Explorations Newsletter, vol. 12, no. 2, pp. 74-82, 2011.
[33]
S. Wang, J. Yang, N. Chen, X. Chen, and Q. Zhang, Human activity recognition with user-free accelerometers in the sensor networks, in Proc. 2005 Int. Neural Networks and Brain Conf., 2005, pp. 1212-1217.
[34]
Y. Wang, J. Lin, M. Annavaram, Q. A. Jacobson, J. Hong, B. Krishnamachari, and N. Sadeh, A framework of energy efficient mobile sensing for automatic user state recognition, in Proc. 7th Int. Conference on Mobile Systems, Applications, and Services Conf., Kraków, Poland, 2009, pp. 179-192.
[35]
P. Siirtola and J. Röning, Recognizing human activities user-independently on smartphones based on accelerometer data, International Journal of Interactive Multimedia & Artificial Intelligence, vol. 1, no. 5, pp. 38-45, 2012.
[36]
E. Miluzzo, N. D. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X. Zheng, and A. T. Campbell, Sensing meets mobile social networks: The design, implementation and evaluation of the cenceme application, in Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems, ACM, 2008, pp. 337-350.
[37]
J. S. Carós, O. Chételat, P. Celka, S. Dasen, and J. CmÃral, Very low complexity algorithm for ambulatory activity classification, presented at the 3rd European Medical and Biological Conference, Prague, Czech, 2005.
[38]
S. R. Garner, Weka: The waikato environment for knowledge analysis, in Proc. 2nd New Zealand Computer Science Research Students Conf., Hamilton, New Zealand, 1995, pp. 57-64.
[39]
J. R. Quinlan, C4. 5: Programs for Machine Learning, vol. 1. Morgan Kaufmann, 1993.
[40]
D. Maguire and R. Frisby, Comparison of feature classification algorithm for activity recognition based on accelerometer and heart rate data, presented at the 9th IT & T Conference, Dubrin, Ireland, 2009.
[41]
H. Martín, A. M. Bernardos, J. Iglesias, and J. R. Casar, Activity logging using lightweight classification techniques in mobile devices, Personal and Ubiquitous Computing, vol. 17, no. 4, pp. 675-695, 2013.
[42]
R. Kohavi, The power of decision tables, in Machine Learning: ECML-95, Springer, 1995, pp. 174-189.
[43]
Wikipedia, k-nearest neighbors algorithm, http://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm, 2014, Mar. 17.
[44]
C. Lombriser, N. B. Bharatula, D. Roggen, and G. Tröster, On-body activity recognition in a dynamic sensor network, in Proc. 2nd Int. Body Area Networks Conf., Florence, Italy, 2007, pp. 17-22.
[45]
M. Kose, O. D. Incel, and C. Ersoy, Online human activity recognition on smart phones, in Workshop on Mobile Sensing: From Smartphones and Wearables to Big Data, Beijing, China, 2012, pp. 11-15.
[46]
P. Zappi, C. Lombriser, T. Stiefmeier, E. Farella, D. Roggen, L. Benini, and G. Tröster, Activity recognition from on-body sensors: Accuracy-power trade-off by dynamic sensor selection, in Wireless Sensor Networks, Springer, 2008, pp. 17-33.
[47]
N. Oliver and E. Horvitz, A comparison of hmms and dynamic bayesian networks for recognizing office activities, in User Modeling, Springer, 2005, pp. 199-209.
[48]
R. K. Ibrahim, E. Ambikairajah, B. G. Celler, and N. H. Lovell, Time-frequency based features for classification of walking patterns, in Proc. 15th Int. Digital Signal Processing Conf., Wales, UK, 2007, pp. 187-190.
[49]
F. R. Allen, E. Ambikairajah, N. H. Lovell, and B. G. Celler, An adapted gaussian mixture model approach to accelerometry-based movement classification using time-domain features, in Proc. 28th Int. Engineering in Medicine and Biology Society Conf., California, USA, 2006, pp. 3600-3603.
[50]
N. Győrbíró, Á. Fábián, and G. Hományi, An activity recognition system for mobile phones, Mobile Networks and Applications, vol. 14, no. 1, pp. 82-91, 2009.
[51]
I. Anderson, J. Maitland, S. Sherwood, L. Barkhuus, M. Chalmers, M. Hall, B. Brown, and H. Muller, Shakra: Tracking and sharing daily activity levels with unaugmented mobile phones, Mobile Networks and Applications, vol. 12, no. 2-3, pp. 185-199, 2007.
[52]
J.-Y. Yang, J.-S. Wang, and Y.-P. Chen, Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers, Pattern Recognition Letters, vol. 29, no. 16, pp. 2213-2220, 2008.
[53]
N. D. Lane, M. Mohammod, M. Lin, X. Yang, H. Lu, S. Ali, A. Doryab, E. Berke, T. Choudhury, and A. Campbell, Bewell: A smartphone application to monitor, model and promote wellbeing, in Proc. 5th Int. ICST Conference on Pervasive Computing Technologies for Healthcare Conf., Dublin, Ireland, 2011, pp. 23-26.
[54]
T. Saponas, J. Lester, J. Froehlich, J. Fogarty, and J. Landay, Ilearn on the iphone: Real-time human activity classification on commodity mobile phones, University of Washington CSE Tech ReportUW-CSE-08-04-02, 2008.
[55]
M. Berchtold, M. Budde, D. Gordon, H. R. Schmidtke, and M. Beigl, Actiserv: Activity recognition service for mobile phones, in Proc. 9th Int. Wearable Computers Conf., Shanghai, China, 2010, pp. 1-8.
[56]
L. Todorovski and S. Džeroski, Combining classifiers with meta decision trees, Machine Learning, 2003, vol. 50, no. 3, pp. 223-249.
[57]
M. Zhang and A. A. Sawchuk, Motion primitive-based human activity recognition using a bag-of-features approach, in Proc. 2nd Int. Health Informatics Symposium, Miami, USA, 2012, pp. 631-640.
[58]
P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features, in Proc. 17th Int. Computer Vision and Pattern Recognition Conf., Kauai, USA, 2001.
[59]
A. Reiss, Personalized mobile physical activity monitoring for everyday life, PhD dissertation. Dept. CS. Technical University of Kaiserslautern, Kaiserslautern, German, 2014.
[60]
W.-Y. Deng, Q.-H. Zheng, and Z.-M. Wang, Cross-person activity recognition using reduced kernel extreme learning machine, Neural Networks, 2014.
[61]
J.-G. Park, A. Patel, D. Curtis, S. Teller, and J. Ledlie, Online pose classification and walking speed estimation using handheld devices, in Proc. 14th Int. Ubiquitous Computing Conf., Pittsburgh, USA, 2012, pp. 113-122.
[62]
M. Khan, S. I. Ahamed, M. Rahman, and R. O. Smith, A feature extraction method for realtime human activity recognition on cell phones, in Proc. 3rd Int. Symposium on Quality of Life Technology Conf., Toroto, Canada, 2007.
[63]
Z. Yan, V. Subbaraju, D. Chakraborty, A. Misra, and K. Aberer, Energy-efficient continuous activity recognition on mobile phones: An activity-adaptive approach, in Proc. 16th Int. Wearable Computers Symposium, Newcastle, UK, 2012, pp. 17-24.
[64]
Y. Liang, X. Zhou, Z. Yu, and B. Guo, Energy-efficient motion related activity recognition on mobile devices for pervasive healthcare, Mobile Networks and Applications, pp. 1-15, 2013.
[65]
D. Guan, W. Yuan, Y.-K. Lee, A. Gavrilov, and S. Lee, Activity recognition based on semisupervised learning, in Proc. 13th Int. Embedded and Real-Time Computing Systems and Applications Conf., Daegu, Korea, 2007, pp. 469-475.
[66]
A. Blum and T. Mitchell, Combining labeled and unlabeled data with co-training, in Proc. 11th Int. Computational Learning Theory Conf., Wisconsin, USA, 1998, pp. 92-100.
[67]
M. Mahdaviani and T. Choudhury, Fast and scalable training of semi-supervised crfs with application to activity recognition, in NIPS, 2007, vol. 20, pp. 977-984.
[68]
J. Xie and M. S. Beigi, A scale-invariant local descriptor for event recognition in 1d sensor signals, in Proc. Int. Multimedia and Expo Conf., New York, USA, 2009, pp. 1226-1229.
[69]
D. G. Lowe, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[70]
A. M. Khan, Human activity recognition using a single tri-axial accelerometer, Ph.D dissertation, Dept. Computer Engineering, Kyung Hee University, Seoul, Korea, 2011.
[71]
Apple: Nike + iPod application, http://www.apple.com/ipod/nike/, 2014.
[72]
Sleepcycle, http://www.sleepcycle.com/, 2014.
[73]
M. Goel, L. Findlater, and J. Wobbrock, Walktype: Using accelerometer data to accomodate situational impairments in mobile touch screen text entry, in Proc. Human Factors in Computing Systems, 2012, pp. 2687-2696.
[74]
P. Marquardt, A. Verma, H. Carter, and P. Traynor, (sp) iphone: Decoding vibrations from nearby keyboards using mobile phone accelerometers, in Proceedings of the 18th ACM Conference on Computer and Communications Security, ACM, 2011, pp. 551-562.
[75]
H. Si, S. J. Kim, N. Kawanishi, and H. Morikawa, A context-aware reminding system for daily activities of dementia patients, in Proc. 27th Int. Distributed Computing Systems Conf., Toroto, Canada, 2007, pp. 50-50.
[76]
W. Song, J. Lee, H. G. Schulzrinne, and B. Lee, Finding 9-1-1 callers in tall buildings, http://academiccommons.columbia.edu/item/ac:156057, 2014.
[77]
L. Mo, S. Liu, R. X. Gao, D. John, J. Staudenmayer, and P. Freedson, Zigbee-based wireless multisensor system for physical activity assessment, in Engineering in Medicine and Biology Society, EMBC, 2011 Annual International Conference of the IEEE, 2011, pp. 846-849.
[78]
Tsinghua Science and Technology
Pages 235-249
Cite this article:
Su X, Tong H, Ji P. Activity Recognition with Smartphone Sensors. Tsinghua Science and Technology, 2014, 19(3): 235-249. https://doi.org/10.1109/TST.2014.6838194

1370

Views

63

Downloads

267

Crossref

N/A

Web of Science

321

Scopus

0

CSCD

Altmetrics

Received: 01 April 2014
Revised: 15 April 2014
Accepted: 17 April 2014
Published: 18 June 2014
© The author(s) 2014
Return