Journal Home > Volume 9 , Issue 1
Computational Visual Media 2023, 9(1): 177-194 https://doi.org/10.1007/s41095-021-0266-0
Research Article | Open Access | Issue | Published: 18 October 2022

AR assistance for efficient dynamic target search

Show Author's Information Zixiang Zhao1,  Jian Wu1,  Lili Wang1( )
State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing, China
Keywords:
augmented reality (AR), guidance, search
Cite this article:
Zhao Z, Wu J, Wang L. AR assistance for efficient dynamic target search. Computational Visual Media, 2023, 9(1): 177-194. https://doi.org/10.1007/s41095-021-0266-0
Download citation
EndNote(RIS)
BibTeX

671

Views

34

Downloads

Citations

0

Crossref

0

WoS

0

Scopus

0

CSCD

Abstract Full text Electronic supplementary material About this article

When searching for a dynamic target in an unknown real world scene, search efficiency is greatly reduced if users lack information about the spatial structure of the scene. Most target search studies, especially in robotics, focus on determining either the shortest path when the target’s position is known, or a strategy to find the target as quickly as possible when the target’s position is unknown. However, the target’s position is often known intermittently in the real world, e.g., in the case of using surveillance cameras. Our goal is to help user find a dynamic target efficiently in the real world when the target’s position is intermittently known. In order to achieve this purpose, we have designed an AR guidance assistance system to provide optimal current directional guidance to users, based on searching a prediction graph. We assume that a certain number of depth cameras are fixed in a real scene to obtain dynamic target’s position. The system automatically analyzes all possible meetings between the user and the target, and generates optimal directional guidance to help the user catch up with the target. A user study was used to evaluate our method, and its results showed that compared to free search and a top-view method, our method significantly improves target search efficiency.


menu
Abstract
Full text
Outline
Electronic supplementary material
About this article

AR assistance for efficient dynamic target search

Show Author's information Zixiang Zhao1, Jian Wu1, Lili Wang1( )
State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing, China

Abstract

When searching for a dynamic target in an unknown real world scene, search efficiency is greatly reduced if users lack information about the spatial structure of the scene. Most target search studies, especially in robotics, focus on determining either the shortest path when the target’s position is known, or a strategy to find the target as quickly as possible when the target’s position is unknown. However, the target’s position is often known intermittently in the real world, e.g., in the case of using surveillance cameras. Our goal is to help user find a dynamic target efficiently in the real world when the target’s position is intermittently known. In order to achieve this purpose, we have designed an AR guidance assistance system to provide optimal current directional guidance to users, based on searching a prediction graph. We assume that a certain number of depth cameras are fixed in a real scene to obtain dynamic target’s position. The system automatically analyzes all possible meetings between the user and the target, and generates optimal directional guidance to help the user catch up with the target. A user study was used to evaluate our method, and its results showed that compared to free search and a top-view method, our method significantly improves target search efficiency.

Keywords: augmented reality (AR), guidance, search

References(61)

[1]
Lynch, K. Reconsidering the image of the city. In: Cities of the Mind. Environment, Development, and Public Policy. Rodwin, L.; Hollister, R. M. Eds. Springer Boston MA, 151–161, 1984.
[2]
LaViola Jr., J. J.; Kruijff, E.; McMahan, R. P.; Bowman, D.; Poupyrev, I. P. 3D User Interfaces: Theory and Practice. Addison-Wesley Professional, 2017.
[3]
Pierce, J. S.; Pausch, R. Navigation with place representations and visible landmarks. In: Proceedings of the IEEE Virtual Reality, 173–288, 2004.
[4]
Steck, S. D.; Mallot, H. A. The role of global and local landmarks in virtual environment navigation. Presence: Teleoperators and Virtual Environments Vol. 9, No. 1, 69–83, 2000.
DOI Google Scholar
[5]
Darken, R. P.; Cevik, H. Map usage in virtual environments: Orientation issues. Proceedings IEEE Virtual Reality 133–140, 1999.
Google Scholar
[6]
Stoakley, R.; Conway, M. J.; Pausch, R. Virtual reality on a WIM: Interactive worlds in miniature. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 265–272, 1995.
DOI
[7]
Darken, R. P.; Sibert, J. L. A toolset for navigation in virtual environments. In: Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology, 157–165, 1993.
DOI
[8]
Grammenos, D.; Filou, M.; Papadakos, P.; Stephanidis, C. Virtual prints: Leaving trails in virtual environments. In: Proceedings of the Workshop on Virtual Environ-ments, 131–138, 2002.
[9]
Chittaro, L.; Ranon, R.; Ieronutti, L. Guiding visitors of Web3D worlds through automatically generated Tours. In: Proceedings of the 8th International Conference on 3D Web Technology, 27–38, 2003.
DOI
[10]
Elmqvist, N.; Tudoreanu, M. E.; Tsigas, P. Tour generation for exploration of 3D virtual environments. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, 207–210, 2007.
DOI
[11]
Wang, M.; Lyu, X. Q.; Li, Y. J.; Zhang, F. L. VR content creation and exploration with deep learning: A survey. Computational Visual Media Vol. 6, No. 1, 3–28, 2020.
DOI Google Scholar
[12]
Henderson, S.; Feiner, S. Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Transactions on Visualization and Computer Graphics Vol. 17, No. 10, 1355–1368, 2011.
DOI Google Scholar
[13]
Webel, S.; Bockholt, U.; Engelke, T.; Gavish, N.; Olbrich, M.; Preusche, C. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems Vol. 61, No. 4, 398–403, 2013.
DOI Google Scholar
[14]
Barakonyi, I.; Schmalstieg, D. Ubiquitous animated agents for augmented reality. In: Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality, 145–154, 2006.
DOI
[15]
Zauner, J.; Haller, M.; Brandl, A.; Hartman, W. Authoring of a mixed reality assembly instructor for hierarchical structures. In: Proceedings of the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, 237–246, 2003.
DOI
[16]
Syberfeldt, A.; Danielsson, O.; Holm, M.; Wang, L. H. Visual assembling guidance using augmented reality. Procedia Manufacturing Vol. 1, 98–109, 2015.
DOI Google Scholar
[17]
Neumann, U.; Majoros, A. Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. In: Proceedings of the IEEE Virtual Reality Annual International Symposium, 4–11, 1998.
[18]
Nassani, A.; Bai, H. D.; Lee, G.; Billinghurst, M. Tag it!: AR annotation using wearable sensors. In: Proceedings of the SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications, 1–4, 2015.
DOI
[19]
Biocca, F.; Tang, A.; Owen, C.; Xiao, F. Attention funnel: Omnidirectional 3D cursor for mobile aug-mented reality platforms. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1115–1122, 2006.
DOI
[20]
Schwerdtfeger, B.; Reif, R.; Günthner, W. A.; Klinker, G. Pick-by-vision: There is something to pick at the end of the augmented tunnel. Virtual Reality Vol. 15, Nos. 2–3, 213–223, 2011.
DOI Google Scholar
[21]
Kasprzak, S.; Komninos, A.; Barrie, P. Feature-based indoor navigation using augmented reality. In: Proceedings of the 9th International Conference on Intelligent Environments, 100–107, 2013.
DOI
[22]
Alnabhan, A.; Tomaszewski, B. INSAR: Indoor navigation system using augmented reality. In: Proceedings of the 6th ACM SIGSPATIAL International Workshop on Indoor Spatial Awareness, 36–43, 2014.
DOI
[23]
Mulloni, A.; Seichter, H.; Schmalstieg, D. Handheld augmented reality indoor navigation with activity-based instructions. In: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, 211–220, 2011.
DOI
[24]
Kim, J.; Jun, H. Vision-based location positioning using augmented reality for indoor navigation. IEEE Transactions on Consumer Electronics Vol. 54, No. 3, 954–962, 2008.
DOI Google Scholar
[25]
Rehman, U.; Cao, S. Augmented-reality-based indoor navigation: A comparative analysis of handheld devices versus google glass. IEEE Transactions on Human-Machine Systems Vol. 47, No. 1, 140–151, 2017.
DOI Google Scholar
[26]
Subakti, H.; Jiang, J. R. A marker-based cyber-physical augmented-reality indoor guidance system for smart campuses. In: Proceedings of the IEEE 18th International Conference on High Performance Computing and Communications; IEEE 14th International Conference on Smart City; IEEE 2nd International Conference on Data Science and Systems, 1373–1379, 2016.
DOI
[27]
Van Diggelen, F. S. T. A-GPS: Assisted GPS, GNSS, and SBAS. Artech House, 2009.
[28]
Want, R.; Hopper, A.; Falcão, V.; Gibbons, J. The active badge location system. ACM Transactions on Information Systems Vol. 10, No. 1, 91–102, 1992.
DOI Google Scholar
[29]
Fukuju, Y.; Minami, M.; Morikawa, H.; Aoyama, T. DOLPHIN: An autonomous indoor positioning system in ubiquitous computing environment. In: Proceedings IEEE Workshop on Software Technologies for Future Embedded Systems, 53–56, 2003.
[30]
Minami, M.; Fukuju, Y.; Hirasawa, K.; Yokoyama, S.; Mizumachi, M.; Morikawa, H.; Aoyama, T. DOLPHIN: A practical approach for implementing a fully distributed indoor ultrasonic positioning system. In: UbiComp 2004: Ubiquitous Computing. Lecture Notes in Computer Science, Vol. 3205. Davies, N.; Mynatt, E. D.; Siio, I. Eds. Springer Berlin Heidelberg, 347–365, 2004.
DOI
[31]
Liu, M. Y.; Liu, K.; Yang, P. P.; Lei, X. K.; Li, H. Bio-inspired navigation based on geomagnetic. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics, 2339–2344, 2013.
DOI
[32]
Rubino, I.; Barberis, C.; Di Chio, L.; Xhembulla, J.; Malnati, G. Enhancing a museum mobile application through user experience design: A comparative analysis. Recent Advances in Electrical & Electronic Engineering 295–300, 2014.
Google Scholar
[33]
Delail, B. A.; Weruaga, L.; Zemerly, M. J. CAViAR: Context aware visual indoor augmented reality for a university campus. In: Proceedings of the IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, 286–290, 2012.
DOI
[34]
O’rourke, J. Art Gallery Theorems and Algorithms, Vol. 57. Oxford University Press, 1987.
[35]
Zeng, R.; Wen, Y. H.; Zhao, W.; Liu, Y. J. View planning in robot active vision: A survey of systems, algorithms, and applications. Computational Visual Media Vol. 6, No. 3, 225–245, 2020.
DOI Google Scholar
[36]
Foraker, J.; Royset, J. O.; Kaminer, I. Search-trajectory optimization: Part I, formulation and theory. Journal of Optimization Theory and Applications Vol. 169, No. 2, 530–549, 2016.
DOI Google Scholar
[37]
Sato, H.; Royset, J. O. Path optimization for the resource-constrained searcher. Naval Research Logistics Vol. 57, No. 5, 422–440, 2010.
DOI Google Scholar
[38]
Kratzke, T. M.; Stone, L. D.; Frost, J. R. Search and rescue optimal planning system. In: Proceedings of the 13th International Conference on Information Fusion, 1–8, 2010.
DOI
[39]
Royset, J. O.; Sato, H. Route optimization for multiple searchers. Naval Research Logistics Vol. 57, No. 8, 701–717, 2010.
DOI Google Scholar
[40]
Koopman, B. O. Search and Screening: General Principles with Historical Applications. Pergamon Press, 1980.
[41]
Martins, G. H. A new branch-and-bound procedure for computing optimal search paths. Technical Report.Naval Postgraduate School Monterey CA, 1993. Availableat https://apps.dtic.mil/sti/citations/ADA265276.
[42]
Lau, H.; Huang, S. D.; Dissanayake, G. Discounted MEAN bound for the optimal searcher path problem with non-uniform travel times. European Journal of Operational Research Vol. 190, No. 2, 383–397, 2008.
DOI Google Scholar
[43]
Sato, H. Path optimization for single and multiple searchers: Models and algorithms. Technical Report.Naval Postgraduate School Monterey CA, 2008. Availableat https://apps.dtic.mil/sti/citations/ADA488991.
[44]
Morin, M.; Abi-Zeid, I.; Lang, P.; Lamontagne, L.; Maupin, P. The optimal searcher path problem with a visibility criterion in discrete time and space. In: Proceedings of the 12th International Conference on Information Fusion, 2217–2224, 2009.
[45]
Peng, H.; Huo, M. L.; Liu, Z. Z.; Xu, W. Simulation analysis of cooperative target search strategies for multiple UAVs. In: Proceedings of the 27th Chinese Control and Decision Conference, 4855–4859, 2015.
DOI
[46]
Hu, J. W.; Xie, L. H.; Xu, J. Vision-based multi-agent cooperative target search. In: Proceedings of the 12th International Conference on Control Automation Robotics & Vision, 895–900, 2012.
DOI
[47]
Perez-Carabaza, S.; Bermudez-Ortega, J.; Besada-Portas, E.; Lopez-Orozco, J. A.; de la Cruz, J. M. A multi-UAV minimum time search planner based on ACOR. In: Proceedings of the Genetic and Evolutionary Computation Conference, 35–42, 2017.
[48]
Meng, W.; He, Z. R.; Su, R.; Yadav, P. K.; Teo, R.; Xie, L. H. Decentralized multi-UAV flight autonomy for moving convoys search and track. IEEE Transactions on Control Systems Technology Vol. 25, No. 4, 1480–1487, 2017.
DOI Google Scholar
[49]
Hu, J. W.; Xie, L. H.; Xu, J.; Xu, Z. Multi-agent cooperative target search. Sensors Vol. 14, No. 6, 9408–9428, 2014.
DOI Google Scholar
[50]
Bhattacharya, S.; Hutchinson, S. On the existence of Nash equilibrium for a two player pursuit-evasion game with visibility constraints. In: Algorithmic Foundation of Robotics VIII. Springer Tracts in Advanced Robotics, Vol. 57. Chirikjian, G. S.; Choset, H.; Morales, M.; Murphey, T. Eds. Springer Berlin Heidelberg, 251–265, 2009.
[51]
Darken, R. P.; Peterson, B. Spatial orientation, wayfinding, and representation. In: Handbook of Virtual Environment Technology. Stanney, K. Ed. CRC Press, 533–558, 2002.
[52]
Kraus, M.; Schäfer, H.; Meschenmoser, P.; Schweitzer, D.; Keim, D. A.; Sedlmair, M.; Fuchs, J. A comparative study of orientation support tools in virtual reality environments with virtual teleportation. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, 227–238, 2020.
DOI
[53]
Microsoft Azure Kinect DK documentation. 2020. Available at https://docs.microsoft.com/en-us/azure/kinect-dk/.
[54]
Cohen, J. Statistical Power Analysis for the Behavioral Sciences. Academic Press, 2013.
DOI
[55]
Sawilowsky, S. S. New effect size rules of thumb. Journal of Modern Applied Statistical Methods Vol. 8, No. 2, 597–599, 2009.
DOI Google Scholar
[56]
Shapiro, S. S.; Wilk, M. B. An analysis of variance test for normality (complete samples). Biometrika Vol. 52, Nos. 3–4, 591–611, 1965.
DOI Google Scholar
[57]
Rey, D.; Neuhäuser, M. Wilcoxon-signed-rank test. In: International Encyclopedia of Statistical Science. Lovric, M. Ed. Springer Berlin Heidelberg, 1658–1659, 2011.
DOI
[58]
Hart, S. G. Nasa-task load index (NASA-TLX); 20 years later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting Vol. 50, No. 9, 904–908, 2006.
DOI Google Scholar
[59]
Huang, J. H.; Yang, S.; Zhao, Z. S.; Lai, Y. K.; Hu, S. M. ClusterSLAM: A SLAM backend for simultaneous rigid body clustering and motion estimation. Computational Visual Media Vol. 7, No. 1, 87–101, 2021.
DOI Google Scholar
[60]
He, Y. H.; Wei, X.; Hong, X. P.; Shi, W. W.; Gong, Y. H. Multi-target multi-camera tracking by tracklet-to-target assignment. IEEE Transactions on Image Processing Vol. 29, 5191–5205, 2020.
DOI Google Scholar
[61]
Li, P.; Zhang, J. B.; Zhu, Z.; Li, Y. W.; Jiang, L.; Huang, G. State-aware re-identification feature for multi-target multi-camera tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 1506–1516, 2019.
DOI
Video
41095_0266_ESM.mp4
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 28 July 2021
Accepted: 21 December 2021
Published: 18 October 2022
Issue date: March 2023

Copyright

© The Author(s) 2022.

Acknowledgements

We sincerely thank the reviewers for their their constructive suggestions and comments. This work was supported by National Key R&D Program of China (2019YFC1521102) and the National Natural Science Foundation of China (61932003 and 61772051).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduc-tion in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Return