Journal Home > Volume 26 , Issue 5

Motion tracking via Inertial Measurement Units (IMUs) on mobile and wearable devices has attracted significant interest in recent years. High-accuracy IMU-tracking can be applied in various applications, such as indoor navigation, gesture recognition, text input, etc. Many efforts have been devoted to improving IMU-based motion tracking in the last two decades, from early calibration techniques on ships or airplanes, to recent arm motion models used on wearable smart devices. In this paper, we present a comprehensive survey on IMU-tracking techniques on mobile and wearable devices. We also reveal the key challenges in IMU-based motion tracking on mobile and wearable devices and possible directions to address these challenges.


menu
Abstract
Full text
Outline
About this article

Inertial Motion Tracking on Mobile and Wearable Devices: Recent Advancements and Challenges

Show Author's information Zhipeng SongZhichao CaoZhenjiang LiJiliang Wang( )Yunhao Liu
School of Software, Tsinghua University, Beijing 100084, China
Department of Computer Science and Engineering, Michigan State University, Michigan, MI 48824, USA
Department of Computer Science, City University of Hong Kong, Hong Kong 999077, China
Department of Automation and the Global Innovation eXchange Institute (GIX), Tsinghua University, Beijing 100084, China

Abstract

Motion tracking via Inertial Measurement Units (IMUs) on mobile and wearable devices has attracted significant interest in recent years. High-accuracy IMU-tracking can be applied in various applications, such as indoor navigation, gesture recognition, text input, etc. Many efforts have been devoted to improving IMU-based motion tracking in the last two decades, from early calibration techniques on ships or airplanes, to recent arm motion models used on wearable smart devices. In this paper, we present a comprehensive survey on IMU-tracking techniques on mobile and wearable devices. We also reveal the key challenges in IMU-based motion tracking on mobile and wearable devices and possible directions to address these challenges.

Keywords: sensor fusion, Inertial Measurement Units (IMUs), motion tracking, accelerometer, gyroscope, magnetometer

References(72)

[1]
A. Altenbuchner, S. Haug, R. Kretschmer, and K. Weber, How to measure physical motion and the impact of individualized feedback in the field of rehabilitation of geriatric trauma patients, Stud. Health Technol. Inform., vol. 248, pp. 226–232, 2018.
[2]
F. Alemuda and F. J. Lin, Gesture-based control in a smart home environment, in Proc. IEEE Int. Conf. Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Exeter, UK, 2017, pp. 784–791.
[3]
Y. Shen, B. Du, W. Xu, C. Luo, B. Wei, L. Cui, and H. Wen, Securing cyber-physical social interactions on wrist-worn devices, ACM Transactions on Sensor Networks, vol. 16, no. 2, pp. 1–22, 2020.
[4]
Z. M. Zhou, C. S. Wu, Z. Yang, and Y. H. Liu, Sensorless sensing with WiFi, Tsinghua Science and Technology, vol. 20, no. 1, pp. 1–6, 2015.
[5]
Q. Shi, S. H. Zhao, X. W. Cui, M. Q. Lu, and M. D. Jia, Anchor self-localization algorithm based on UWB ranging and inertial measurements, Tsinghua Science and Technology, vol. 24, no. 6, pp. 728–737, 2019.
[6]
S. Jiang, Z. Li, and P. Zhou. Memento: An emotion-driven lifelogging system with wearables, ACM Transactions on Sensor Networks, vol. 15, no. 1, pp. 1–23, 2019.
[7]
Z. Zhang, X. Cong, W. Feng, H. P. Zhang, G. D. Fu, and J. Y. Chen, WAEAS: An optimization scheme of EAS scheduler for wearable applications, Tsinghua Science and Technology, vol. 26, no. 1, pp. 72–84, 2021.
[8]
M. Kok, J. D. Hol, and T. B. Schön, Using inertial sensors for position and orientation estimation, arXiv preprint arXiv: 1704.06053, 2017.
[9]
Z. Yang, C. S. Wu, Z. M. Zhou, X. L. Zhang, X. Wang, and Y. H. Liu, Mobility increases localizability: A survey on wireless indoor localization using inertial sensors, ACM Comput. Surv., vol. 47, no. 3, p. 54, 2015.
[10]
H. Liu, R. Li, S. C. Liu, S. B. Tian, and J. Z. Du, Smartcare: Energy-efficient long-term physical activity tracking using smartphones, Tsinghua Science and Technology, vol. 20, no. 4, pp. 348–363, 2015.
[11]
S. Shen, M. Gowda, and R. R. Choudhury, Closing the gaps in inertial motion tracking, in Proc. 24th Annu. Int. Conf. Mobile Computing and Networking (MobiCom), New Delhi, India, 2018, pp. 429–444.
[12]
O. J. Woodman, An introduction to inertial navigation, Technical Report UCAM-CL-TR-696, Computer Laboratory, University of Cambridge, https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-696.html, 2007.
[13]
G. Laput, R. Xiao, and C. Harrison, ViBand: High-fidelity bio-acoustic sensing using commodity smartwatch accelerometers, in Proc. 29th Annu. Symp. User Interface Software and Technology (UIST), New York, NY, USA, 2016, pp. 321–333.
[14]
V. T. Van Hees, L. Gorzelniak, E. C. D. León, M. Eder, M. Pias, S. Taherian, U. Ekelund, F. Renström, P. W. Franks, A. Horsch, et al., Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity, PLoS One, vol. 8, no. 4, p. e61691, 2013.
[15]
Android developer, Motion sensors, https://developer.android.com/guide/topics/sensors/sensors_motion, 2021.
[16]
P. F. Zhou, M. Li, and G. B. Shen, Use it free: Instantly knowing your phone attitude, in Proc. 20th Annu. Int. Conf. Mobile Computing and Networking (MobiCom), Maui, HI, USA, 2014, pp. 605–616.
[17]
S. Poddar, V. Kumar, and A. Kumar, A comprehensive overview of inertial sensor calibration techniques, J. Dyn. Syst. Meas. Control, vol. 139, no. 1, p. 011006, 2016.
[18]
P. D. Groves, Principles of GNSS, Inertial, and Multisensor Integrated Navigation Systems. 2nd ed. London, UK: Artech House, 2013.
[19]
M. J. Caruso, Applications of magnetic sensors for low cost compass systems, in Proc. IEEE 2000 Position Location and Navigation Symp., San Diego, CA, USA, 2000, pp. 177–184.
[20]
Y. F. Yin, L. Xie, T. Gu, Y. J. Lu, and S. L. Lu, Aircontour: Building contour-based model for in-air writing gesture recognition, ACM Trans. Sensor Net., vol. 15, no. 4, p. 44, 2019.
[21]
P. Batista, C. Silvestre, P. Oliveira, and B. Cardeira, Accelerometer calibration and dynamic bias and gravity estimation: Analysis, design, and experimental evaluation, IEEE Trans. Control Syst. Technol., vol. 19, no. 5, pp. 1128–1137, 2011.
[22]
E. H. Shin and E. S. Naser, Accuracy improvement of low cost INS/GPS for land applications, in Proc. 2002 National Technical Meeting of the Institute of Navigation, San Diego, CA, USA, 2002, pp. 146–157.
[23]
E. H. Shin and N. El-Sheimy, A new calibration method for strapdown inertial navigation systems, Zfv, vol. 127, pp. 41–50, 2002.
[24]
X. J. Niu, Y. Li, H. P. Zhang, Q. J. Wang, and Y. L. Ban, Fast thermal calibration of low-grade inertial sensors and inertial measurement units, Sensors, vol. 13, no. 9, pp. 12192–12217, 2013.
[25]
J. L. Crassidis, F. L. Markley, and Y. Cheng, Survey of nonlinear attitude estimation methods, J. Guid. Control Dyn., vol. 30, no. 1, pp. 12–28, 2007.
[26]
N. H. Q. Phuong, H. J. Kang, Y. S. Suh, and Y. S. Ro, A DCM based orientation estimation algorithm with an inertial measurement unit and a magnetic compass, J. Univ. Comp. Sci., vol. 15, no. 4, pp. 859–876, 2009.
[27]
E. Edwan, J. Y. Zhang, J. C. Zhou, and O. Loffeld, Reduced DCM based attitude estimation using low-cost IMU and magnetometer triad, in Proc. 2011 8th Workshop on Positioning, Navigation and Communication, Dresden, Germany, 2011, pp. 1–6.
[28]
S. Yean, B. S. Lee, C. K. Yeo, and C. H. Vun, Algorithm for 3D orientation estimation based on Kalman filter and gradient descent, in Proc. 2016 IEEE 7th Annu. Information Technology, Electronics and Mobile Communication Conf. (IEMCON), Vancouver, Canada, 2016, pp. 1–6.
[29]
L. Lou, X. Xu, J. Cao, Z. L. Chen, and Y. Xu, Sensor fusion-based attitude estimation using low-cost MEMS-IMU for mobile robot navigation, in Proc. 2011 6th IEEE Joint Int. Information Technology and Artificial Intelligence Conf., Chongqing, China, 2011, pp. 465–468.
[30]
W. Li and J. L. Wang, Effective adaptive Kalman filter for MEMS-IMU/magnetometers integrated attitude and heading reference systems, J. Navigat., vol. 66, no. 1, pp. 99–113, 2013.
[31]
R. Zhu, D. Sun, Z. Y. Zhou, and D. Q. Wang, A linear fusion algorithm for attitude determination using low cost mems-based sensors, Measurement, vol. 40, no. 3, pp. 322–328, 2007.
[32]
D. Jurman, M. Jankovec, R. Kamnik, and M. Topic, Calibration and data fusion solution for the miniature attitude and heading reference system, Sens. Actuators A: Phys., vol. 138, no. 2, pp. 411–420, 2007.
[33]
Z. Ercan, V. Sezer, H. Heceoglu, C. Dikilitas, M. Gokasan, A. Mugan, and S. Bogosyan, Multi-sensor data fusion of DCM based orientation estimation for land vehicles, in Proc. 2011 IEEE Int. Conf. Mechatronics, Istanbul, Turkey, 2011, pp. 672–677.
[34]
N. Shantha and T. Jann, Estimation of attitudes from a low-cost miniaturized inertial platform using kalman filter-based sensor fusion algorithm, Sadhana, vol. 29, no. 2, pp. 217–235, 2004.
[35]
H. F. Grip, T. I. Fossen, T. A. Johansen, and A. Saberi, Globally exponentially stable attitude and gyro bias estimation with application to GNSS/INS integration, Automatica, vol. 51, pp. 158–166, 2015.
[36]
F. Olsson, M. Kok, K. Halvorsen, and T. B. Schön, Accelerometer calibration using sensor fusion with a gyroscope, in Proc. 2016 IEEE Statistical Signal Proc. Workshop (SSP), Palma de Mallorca, Spain, 2016, pp. 1–5.
[37]
E. Foxlin, Inertial head-tracker sensor fusion by a complementary separate-bias Kalman filter, in Proc. IEEE 1996 Virtual Reality Annu. Int. Symp., Santa Clara, CA, USA, 1996, pp. 185–194.
[38]
R. Munguia and A. Grau, Attitude and heading system based on EKF total state configuration, in Proc. 2011 IEEE Int. Symp. Industrial Electronics, Gdansk, Poland, 2011, pp. 2147–2152.
[39]
M. Romanovas, L. Klingbeil, M. Trachtler, and Y. Manoli, Efficient orientation estimation algorithm for low cost inertial and magnetic sensor systems, in Proc. 2009 IEEE/SP 15th Workshop on Statistical Signal Proc., Cardiff, UK, 2009, pp. 586–589.
[40]
H. G. de Marina, F. J. Pereda, J. M. Giron-Sierra, and F. Espinosa, UAV attitude estimation using unscented Kalman filter and TRIAD, IEEE Trans. Indust. Electron., vol. 59, no. 11, pp. 4465–4474, 2012.
[41]
J. L. Marins, X. P. Yun, E. R. Bachmann, R. B. McGhee, and M. J. Zyda, An extended Kalman filter for quaternion-based orientation estimation using MARG sensors, in Proc. 2001 IEEE/RSJ Int. Conf. Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the Next Millennium (Cat. No. 01CH37180), Maui, HI, USA, 2002, pp. 2003–2011.
[42]
A. M. Sabatini, Quaternion-based extended Kalman filter for determining orientation by inertial and magnetic sensing, IEEE Trans. Biomed. Eng., vol. 53, no. 7, pp. 1346–1356, 2006.
[43]
J. Goslinski, M. Nowicki, and P. Skrzypczynski, Performance comparison of EKF-based algorithms for orientation estimation on android platform, IEEE Sens.J., vol. 15, no. 7, pp. 3781–3792, 2015.
[44]
B. Huyghe, J. Doutreloigne, and J. Vanfleteren, 3D orientation tracking based on unscented Kalman filtering of accelerometer and magnetometer data, in Proc. 2009 IEEE Sensors Applications Symp, New Orleans, LA, USA, 2009, pp. 148–152.
[45]
R. Mahony, T. Hamel, and J. M. Pflimlin, Nonlinear complementary filters on the special orthogonal group, IEEE Trans. Automat. Control, vol. 53, no. 5, pp. 1203–1218, 2008.
[46]
M. Nowicki, J. Wietrzykowski, and P. Skrzypczynski, Simplicity or flexibility? complementary filter vs. EKF for orientation estimation on mobile devices, in Proc. 2015 IEEE 2nd Int. Conf. Cybernetics (CYBCONF), Gdynia, Poland, 2015, pp. 166–171.
[47]
S. O. H. Madgwick, An Efficient Orientation Filter for Inertial and Inertial/Magnetic Sensor Arrays. UK: University of Bristol, 2010.
[48]
S. O. H. Madgwick, A. J. L. Harrison, and R. Vaidyanathan, Estimation of IMU and MARG orientation using a gradient descent algorithm, in Proc. 2011 IEEE Int. Conf. Rehabilitation Robotics, Zurich, Switzerland, 2011, pp. 1–7.
[49]
M. El-Gohary, S. Pearson, and J. McNames, Joint angle tracking with inertial sensors, Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., vol. 208, pp. 1068–1071, 2008.
[50]
M. El-Gohary and J. McNames, Shoulder and elbow joint angle tracking with inertial sensors, IEEE Trans. Biomed. Eng., vol. 59, no. 9, pp. 2635–2641, 2012.
[51]
S. Shen, H. Wang, and R. R. Choudhury, I am a smartwatch and I can track my user’s arm, in Proc. 14th Annu. Int. Conf. Mobile Systems, Applications, and Services, MobiSys’16, Singapore, 2016, pp. 85–96.
[52]
Y. Liu, C. D. Lin, Z. J. Li, Z. D. Liu, and K. S. Wu, When wearable sensing meets arm tracking (poster), in Proc. 17th Annu. Int. Conf. Mobile Systems, Applications, and Services, MobiSys’19, Seoul, Republic of Korea, 2019, pp. 518–519.
[53]
B. Fang, X. Wei, F. C. Sun, H. M. Huang, Y. L. Yu, and H. P. Liu, Skill learning for human-robot interaction using wearable device, Tsinghua Sci. Technol., vol. 24, no. 6, pp. 654–662, 2019.
[54]
P. del Moral, Nonlinear filtering: Interacting particle solution, Markov Process. Relat. Fields, vol. 2, no. 4, pp. 555–580, 1996.
[55]
X. Y. Liu, Z. Zhou, W. R. Diao, Z. Li, and K. H. Zhang, When good becomes evil: Keystroke inference with smartwatch, in Proc. 22nd ACM SIGSAC Conf. Computer and Communications Security, Denver, CO, USA, 2015, pp. 1273–1285.
[56]
C. Xu, P. H. Pathak, and P. Mohapatra, Finger-writing with smartwatch: A case for finger and hand gesture recognition using smartwatch, in Proc.16th Int. Workshop on Mobile Computing Systems and Applications, Santa Fe, NM, USA, 2015, pp. 9–14.
[57]
Z. J. Ba, T. H. Zheng, X. Y. Zhang, Z. Qin, B. C. Li, X. Liu, and K. Ren, Learning-based practical smartphone eavesdropping with built-in accelerometer, in Proc. Network and Distributed Systems Security (NDSS) Symp., San Diego, CA, USA, 2020, pp. 23–26.
[58]
J. H. Hou, X. Y. Li, P. D. Zhu, Z. F. Wang, Y. Wang, J. W. Qian, and P. L. Yang, SignSpeaker: A real-time, high-precision smartwatch-based sign language translator, in Proc. 25th Annu. Int. Conf. Mobile Computing and Networking, MobiCom’19, Los Cabos, Mexico, 2019, pp. 24:1–24:15.
[59]
A. Parate, M. C. Chiu, C. Chadowitz, D. Ganesan, and E. Kalogerakis, RisQ: Recognizing smoking gestures with inertial sensors on a wristband, in Proc. 12th Annu. Int. Conf. Mobile Systems, Applications, and Services, MobiSys’14, Bretton Woods, NH, USA, 2014, pp. 149–161.
[60]
C. Amma, M. Georgi, and T. Schultz, Airwriting: A wearable handwriting recognition system, Pers. Ubiquit. Comput., vol. 18, no. 1, pp. 191–203, 2014.
[61]
S. D. Choi, A. S. Lee, and S. Y. Lee, On-line handwritten character recognition with 3D accelerometer, in Proc. 2006 IEEE Int. Conf. Information Acquisition, Veihai, China, 2006, pp. 845–850.
[62]
J. H. Wu, G. Pan, D. Q. Zhang, G. D. Qi, and S. J. Li, Gesture recognition with a 3-D accelerometer, in Proc. 6th Int. Conf. Ubiquitous Intelligence and Computing (UIC) 2009, Heidelberg, Germany, 2009, pp. 25–38.
[63]
L. Sun, D. Q. Zhang, B. Li, B. Guo, and S. J. Li, Activity recognition on an accelerometer embedded mobile phone with varying positions and orientations, in Proc. 7th Int. Conf. Ubiquitous Intelligence and Computing (UIC), Berlin, Germany, 2010, pp. 548–562.
[64]
Xia Q. X., Hong F., Feng Y., and Guo Z. W., Motionhacker: Motion sensor-based eavesdropping on handwriting via smartwatch, in Proc. IEEE INFOCOM 2018-IEEE Conf. Computer Communications Workshops (INFOCOM WKSHPS), Honolulu, HI, USA, 2018, pp. 468473.
[65]
S. Sen, K. Grover, V. Subbaraju, and A. Misra, Inferring smartphone keypress via smartwatch inertial sensing, in Proc. 2017 IEEE Int. Conf. Pervasive Computing and Communications Workshops (PerCom Workshops), Kona, HI, USA, 2017, pp. 685–690.
[66]
C. Zhang, J. R.Yang, C. Southern, T. E. Starner, and G. D. Abowd, Watchout: Extending interactions on a smartwatch with inertial sensing, in Proc. ACM Int. Symp. Wearable Computers, Kobe, Japan, 2016, pp. 136–143.
[67]
T. H. Vu, A. Misra, Q. Roy, K. C. T. Wei, and Y. Lee, Smartwatch-based early gesture detection 8 trajectory tracking for interactive gesture-driven applications, Proc. ACM Interact. Mobile Wearab. Ubiquit. Technol., vol. 2, no. 1, p. 39, 2018.
[68]
H. Wang, T. T. T. Lai, and R. R. Choudhury, MoLe: Motion leaks through smartwatch sensors, in Proc. 21stAnnu. Int. Conf. Mobile Computing and Networking, MobiCom’15, Paris, France, 2015, pp. 155–166.
[69]
S. Agrawal, I. Constandache, S. Gaonkar, R. R. Choudhury, K. Caves, and F. DeRuyter, Using mobile phones to write in air, in Proc. 9th Int. Conf. Mobile Systems, Applications, and Services, MobiSys’11, Bethesda, MD, USA, 2011, pp. 15–28.
[70]
K. Katsuragawa, J. R. Wallace, and E. Lank, Gestural text input using a smartwatch, in Proc. Working Conf. Advanced Visual Interfaces, Bari, Italy, 2016, pp. 220–223.
[71]
L. Ardüser, P. Bissig, P. Brandes, and R. Wattenhofer, Recognizing text using motion data from a smartwatch, in Proc. 2016 IEEE Int. Conf. Pervasive Computing and Communication Workshops (PerCom Workshops), Sydney, Australia, 2016, pp. 1–6.
[72]
T. Deselaers, D. Keysers, J. Hosang, and H. A. Rowley, Gyropen: Gyroscopes for pen-input with mobile phones, IEEE Trans. Human Mach. Syst., vol. 45, no. 2, pp. 263–271, 2015.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 07 February 2021
Accepted: 26 February 2021
Published: 20 April 2021
Issue date: October 2021

Copyright

© The author(s) 2021

Acknowledgements

This work was in part supported by the National Key R&D Program of China (No. 2018YFB1004800) and the National Natural Science Foundation of China (No. 61932013).

Rights and permissions

© The author(s) 2021. The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return