Journal Home > Volume 28 , Issue 3
Tsinghua Science and Technology 2023, 28(3): 552-565 https://doi.org/10.26599/TST.2022.9010029
Open Access | Issue | Published: 13 December 2022

From Coarse to Fine: Two-Stage Indoor Localization with Multisensor Fusion

Show Author's Information Li Zhang1( ),  Jinhui Bao1,  Yi Xu1,  Qiuyu Wang1,  Jingao Xu2,  Danyang Li2
School of Mathematics, Hefei University of Technology, Hefei 230009, China
School of Software and BNRist, Tsinghua University, Beijing 100084, China
Keywords:
computer vision, indoor localization, WiFi fingerprints, Pedestrian Dead Reckoning (PDR)
Cite this article:
Zhang L, Bao J, Xu Y, et al. From Coarse to Fine: Two-Stage Indoor Localization with Multisensor Fusion. Tsinghua Science and Technology, 2023, 28(3): 552-565. https://doi.org/10.26599/TST.2022.9010029
Download citation
EndNote(RIS)
BibTeX

458

Views

44

Downloads

Citations

1

Crossref

1

WoS

1

Scopus

0

CSCD

Abstract Full text About this article

Increasing attention has been paid to high-precision indoor localization in dense urban and indoor environments. Previous studies have shown single indoor localization methods based on WiFi fingerprints, surveillance cameras or Pedestrian Dead Reckoning (PDR) are restricted by low accuracy, limited tracking region, and accumulative error, etc., and some defects can be resolved with more labor costs or special scenes. However, requesting more additional information and extra user constraints is costly and rarely applicable. In this paper, a two-stage indoor localization system is presented, integrating WiFi fingerprints, the vision of surveillance cameras, and PDR (the system abbreviated as iWVP). A coarse location using WiFi fingerprints is done advanced, and then an accurate location by fusing data from surveillance cameras and the IMU sensors is obtained. iWVP uses a matching algorithm based on motion sequences to confirm the identity of pedestrians, enhancing output accuracy and avoiding corresponding drawbacks of each subsystem. The experimental results show that the iWVP achieves high accuracy with an average position error of 4.61 cm, which can effectively track pedestrians in multiple regions in complex and dynamic indoor environments.


menu
Abstract
Full text
Outline
About this article

From Coarse to Fine: Two-Stage Indoor Localization with Multisensor Fusion

Show Author's information Li Zhang1( ), Jinhui Bao1, Yi Xu1, Qiuyu Wang1, Jingao Xu2, Danyang Li2
School of Mathematics, Hefei University of Technology, Hefei 230009, China
School of Software and BNRist, Tsinghua University, Beijing 100084, China

Abstract

Increasing attention has been paid to high-precision indoor localization in dense urban and indoor environments. Previous studies have shown single indoor localization methods based on WiFi fingerprints, surveillance cameras or Pedestrian Dead Reckoning (PDR) are restricted by low accuracy, limited tracking region, and accumulative error, etc., and some defects can be resolved with more labor costs or special scenes. However, requesting more additional information and extra user constraints is costly and rarely applicable. In this paper, a two-stage indoor localization system is presented, integrating WiFi fingerprints, the vision of surveillance cameras, and PDR (the system abbreviated as iWVP). A coarse location using WiFi fingerprints is done advanced, and then an accurate location by fusing data from surveillance cameras and the IMU sensors is obtained. iWVP uses a matching algorithm based on motion sequences to confirm the identity of pedestrians, enhancing output accuracy and avoiding corresponding drawbacks of each subsystem. The experimental results show that the iWVP achieves high accuracy with an average position error of 4.61 cm, which can effectively track pedestrians in multiple regions in complex and dynamic indoor environments.

Keywords: computer vision, indoor localization, WiFi fingerprints, Pedestrian Dead Reckoning (PDR)

References(44)

[1]
H. Motte, J. Wyffels, L. De Strycker, and J. P. Goemaere, Evaluating GPS data in indoor environments, Adv. Electr. Computer Eng., vol. 11, no. 3, pp. 25–28, 2011.
DOI Google Scholar
[2]
C. S. Wu, J. G. Xu, Z. Yang, N. D. Lane, and Z. W. Yin, Gain without pain: Accurate WiFi-based localization using fingerprint spatial gradient, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 2, pp. 1–19, 2017.
DOI Google Scholar
[3]
C. S. Wu, Z. Yang, C. W. Xiao, C. F. Yang, Y. H. Liu, and M. Y. Liu, Static power of mobile devices: Self-updating radio maps for wireless indoor localization, in Proc. 2015 IEEE Conf. Computer Communications (INFOCOM), Hong Kong, China, 2015, pp. 2497–2505.
DOI Google Scholar
[4]
Z. Yang, C. S. Wu, and Y. H. Liu, Locating in fingerprint space: Wireless indoor localization with little human intervention, in Proc. 18th Ann. Int. Conf. Mobile Computing and Networking, Istanbul, Turkey, 2012, pp. 269–280.
DOI Google Scholar
[5]
P. Dollár, R. Appel, and W. Kienzle, Crosstalk Cascades for Frame-rate Pedestrian Detection. Berlin, Germany: Springer, 2012.
DOI
[6]
B. Y. Liu, J. Z. Huang, Y. Lin, and C. Kulikowski, Robust tracking using local sparse appearance model and K-selection, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 2011, pp. 1313–1320.
Google Scholar
[7]
H. Pirsiavash, D. Ramanan, and C. C. Fowlkes, Globallyoptimal greedy algorithms for tracking a variable number of objects, in Proc. 2011 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2011, pp. 1210–1208.
DOI Google Scholar
[8]
A. Rai, K. K. Chintalapudi, V. N. Padmanabhan, and R. Sen, Zee: Zero-effort crowdsourcing for indoor localization, in Proc. 18th Ann. Int. Conf. Mobile Computing and Networking, Istanbul, Turkey, 2012, pp. 293–304.
DOI Google Scholar
[9]
C. S. Wu, Z. Yang, and C. W. Xiao, Automatic radio map adaptation for indoor localization using smartphones, IEEE Trans. Mobile Comput., vol. 17, no. 3, pp. 517–528, 2017.
DOI Google Scholar
[10]
Z. Yang, C. S. Wu, Z. M. Zhou, X. L. Zhang, X. Wang, and Y. H. Liu, Mobility increases localizability: A survey on wireless indoor localization using inertial sensors, ACM Comput. Surv., vol. 47, no. 3, p. 54, 2015.
DOI Google Scholar
[11]
Q. Shi, S. H. Zhao, X. W. Cui, M. Q. Lu, and M. D. Jia, Anchor self-localization algorithm based on UWB ranging and inertial measurements, Tsinghua Science and Technology, vol. 24, no. 6, pp. 728–737, 2019.
DOI Google Scholar
[12]
Q. Z. Lin and Y. Guo, Accurate indoor navigation system using human-item spatial relation, Tsinghua Science and Technology, vol. 21, no. 5, pp. 521–537, 2016.
DOI Google Scholar
[13]
Q. X. Chen, D. D. Ding, and Y. Zheng, Indoor pedestrian tracking with sparse RSS fingerprints, Tsinghua Science and Technology, vol. 23, no. 1, pp. 95–103, 2018.
DOI Google Scholar
[14]
L. M. Ni, Y. H. Liu, Y. C. Lau, and A. Patil, LANDMARC: Indoor location sensing using active RFID, in Proc. 1st IEEE Int. Conf. Pervasive Computing and Communications, Fort Worth, TX, USA, 2003, pp. 407–415.
Google Scholar
[15]
Y. H. Liu, J. L. Wang, Y. T. Zhang, L. S. Cheng, W. Y. Wang, Z. Wang, W. M. Xu, and Z. J. Li, Vernier: Accurate and fast acoustic motion tracking using mobile devices, IEEE Trans. Mobile Comput., vol. 20, no. 2, pp. 754–764, 2021.
DOI Google Scholar
[16]
S. G. Wei, J. K. Wang, and Z. H. Zhao, Poster abstract: LocTag: Passive WiFi tag for robust indoor localization via smartphones, in Proc. IEEE INFOCOM 2020-IEEE Conf. Computer Communications Workshops (INFOCOM WKSHPS), Toronto, Canada, 2020, pp. 1342–1343.
DOI Google Scholar
[17]
S. Saloni and A. Hegde, WiFi-aware as a connectivity solution for IoT pairing IoT with WiFi aware technology: Enabling new proximity based services, in Proc. 2016 Int. Conf. Internet of Things and Applications (IOTA), Pune, India, 2016, pp. 137–142.
DOI Google Scholar
[18]
W. Gong and J. C. Liu, SiFi: Pushing the limit of time-based WiFi localization using a single commodity access point, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 2, no. 1, p. 10, 2018.
DOI Google Scholar
[19]
B. Shin, S. Lee, C. Kim, J. Kim, T. Lee, C. Kee, S. Heo, and H. Rhee, Implementation and performance analysis of smartphone-based 3D PDR system with hybrid motion and heading classifier, in Proc. 2014 IEEE/ION Position, Location and Navigation Symp.-PLANS 2014, Monterey, CA, USA, 2014, pp. 201–204.
DOI Google Scholar
[20]
X. C. Liu, Y. R. Jiang, P. Jain, and K. H. Kim, TAR: Enabling fine-grained targeted advertising in retail stores, in Proc. 16th Ann. Int. Conf. Mobile Systems, Applications, and Services, Munich, Germany, 2018, pp. 323–336.
DOI Google Scholar
[21]
J. Teng, B. Y. Zhang, J. D. Zhu, X. F. Li, D. Xuan, and Y. F. Zheng, EV-Loc: Integrating electronic and visual signals for accurate localization, IEEE/ACM Trans. Network., vol. 22, no. 4, pp. 1285–1296, 2013.
DOI Google Scholar
[22]
W. Ma, Q. Q. Li, B. D. Zhou, W. X. Xue, and Z. D. Huang, Location and 3-D visual awareness-based dynamic texture updating for indoor 3-D model, IEEE Internet Things J., vol. 7, no. 8, pp. 7612–7624, 2020.
DOI Google Scholar
[23]
J. X. Gu, J. L. Wang, L. Zhang, Z. W. Yu, X. Z. Xin, and Y. H. Liu, Spotlight: Hot target discovery and localization with crowdsourced photos, Tsinghua Science and Technology, vol. 25, no. 1, pp. 68–80, 2020.
DOI Google Scholar
[24]
X. Q. Teng, D. K. Guo, Y. L. Guo, X. L. Zhou, and Z. Liu, CloudNavi: Toward ubiquitous indoor navigation service with 3D point clouds, ACM Trans. Sensor Networks, vol. 15, no. 1, p. 1, 2019.
DOI Google Scholar
[25]
A. R. Jimenez, F. Seco, C. Prieto, and J. Guevara, A comparison of pedestrian dead-reckoning algorithms using a low-cost MEMS IMU, in Proc. 2009 IEEE Int. Symp. Intelligent Signal Proc., Budapest, Hungary, 2009, pp. 37–42.
DOI Google Scholar
[26]
Z. D. Li, Z. B. Su, and T. T. Yang, Design of intelligent mobile robot positioning algorithm based on IMU/Odometer/Lidar, in Proc. 2019 Int. Conf. Sensing, Diagnostics, Prognostics, and Control (SDPC), Beijing, China, 2019, pp. 627–631.
DOI Google Scholar
[27]
R. Harle, A survey of indoor inertial positioning systems for pedestrians, IEEE Commun. Surv. Tut., vol. 15, no. 3, pp. 1281–1293, 2013.
DOI Google Scholar
[28]
M. Q. Zhan and Z. H. Xi, Indoor location method of WiFi/PDR fusion based on extended Kalman filter fusion, J. Phys.: Conf. Ser., vol. 1601, no. 4, p. 042004, 2020.
DOI Google Scholar
[29]
J. G. Xu, H. J. Chen, K. Qian, E. Q. Dong, M. Sun, C. S. Wu, L. Zhang, and Z. Yang, iVR: Integrated vision and radio localization with zero human effort, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 3, no. 3, p. 114, 2019.
DOI Google Scholar
[30]
K. S. Wu, J. Xiao, Y. W. Yi, M. Gao, and L. M. Ni, FILA: Fine-grained indoor localization, in Proc. 2012 IEEE INFOCOM, Orlando, FL, USA, 2012, pp. 2210–2218.
DOI Google Scholar
[31]
P. Bahl and V. N. Padmanabhan, RADAR: An in-building RF-based user location and tracking system, in Proc. IEEE INFOCOM 2000. Conf. Computer Communications. Nineteenth Ann. Joint Conf. IEEE Computer and Communications Societies (Cat. No. 00CH37064), Tel Aviv, Israel, 2000, pp. 775–784.
Google Scholar
[32]
M. Youssef and A. Agrawala, The Horus WLAN location determination system, in Proc. 3rd Int. Conf. Mobile Systems, Applications, and Services, Washington, DC, USA, 2005, pp. 205–218.
DOI Google Scholar
[33]
H. F. Yang, Y. B. Zhang, Y. L. Huang, H. M. Fu, and Z. H. Wang, WKNN indoor location algorithm based on zone partition by spatial features and restriction of former location, Pervasive Mob. Comput., vol. 60, p. 101085, 2019.
DOI Google Scholar
[34]
Y. F. Le, H. N. Zhang, W. B. Shi, and H. Yao, Received signal strength based indoor positioning algorithm using advanced clustering and kernel ridge regression, Front. Inform. Technol. Electron. Eng., vol. 22, no. 6, pp. 827–838, 2021.
DOI Google Scholar
[35]
J. G. Xu, Y. Zheng, H. J. Chen, Y. H. Liu, X. C. Zhou, J. B. Li, and N. Lane, Embracing spatial awareness for reliable WiFi-based indoor location systems, in Proc. 2018 IEEE 15th Int. Conf. Mobile Ad Hoc and Sensor Systems (MASS), Chengdu, China, 2018, pp. 281–289.
DOI Google Scholar
[36]
D. G. Lowe, Distinctive image features from scale-invariant keypoints, Int.J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004.
DOI Google Scholar
[37]
H. Bay, T. Tuytelaars, and L. Van Gool, SURF: Speeded up robust features, in Proc. 9th European Conf. Computer Vision-Volume Part I, Graz, Austria, 2006, pp. 404–417.
DOI Google Scholar
[38]
J. J. Yan, G. G. He, A. Basiri, and C. Hancock, 3-D passive-vision-aided pedestrian dead reckoning for indoor positioning, IEEE Trans. Instrum. Meas., vol. 69, no.4, pp. 1370–1386, 2020.
DOI Google Scholar
[39]
X. L. Gan, B. G. Yu, H. Zhang, L. Huang, and Y. N. Li, Indoor combination positioning technology of pseudolites and PDR, in Proc. 2018 Ubiquitous Positioning, Indoor Navigation and Location-Based Services, Wuhan, China, 2018, pp. 1–7.
DOI Google Scholar
[40]
A. Poulose and D. S. Han, Indoor localization using PDR with Wi-Fi weighted path loss algorithm, in Proc. 2019 Int. Conf. Information and Communication Technology Convergence (ICTC), Jeju, Republic of Korea, 2019, pp. 689–693.
DOI Google Scholar
[41]
Z. H. Chen, H. Zou, H. Jiang, Q. C. Zhu, Y. C. Soh, and L. H. Xie, Fusion of WiFi, smartphone sensors and landmarks using the Kalman filter for indoor localization, Sensors, vol. 15, no. 1, pp. 715–732, 2015.
DOI Google Scholar
[42]
D. Y. Li, Y. M. Lu, J. G. Xu, Q. Ma, and Z. Liu, iPAC: Integrate pedestrian dead reckoning and computer vision for indoor localization and tracking, IEEE Access, vol. 7, pp. 183514–183523, 2019.
DOI Google Scholar
[43]
D. N. Fernández, Implementation of a WiFi-based indoor location system on a mobile device for a university area, in Proc. 2019 IEEE XXVI Int. Conf. Electronics, Electrical Engineering and Computing (INTERCON), Lima, Peru, 2019, pp. 1–4.
DOI Google Scholar
[44]
L. Chruszczyk, Statistical analysis of indoor RSSI readouts for 433 MHz, 868 MHz, 2.4 GHz and 5 GHz ISM bands, Int.J. Electron. Telec., vol. 63, no. 1, pp. 33–38, 2017.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 19 March 2022
Revised: 29 June 2022
Accepted: 08 August 2022
Published: 13 December 2022
Issue date: June 2023

Copyright

© The author(s) 2023.

Acknowledgements

This work was supported by the National Key Research and Development Program (No. 2018YFB2100301) and the National Natural Science Foundation of China (No. 61972131).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return