Journal Home > Volume 5 , Issue 3
Journal of Intelligent and Connected Vehicles 2022, 5(3): 215-234 https://doi.org/10.1108/JICV-05-2022-0017
Research paper | Open Access | Issue | Published: 12 July 2022

Analyzing the inconsistency in driving patterns between manual and autonomous modes under complex driving scenarios with a VR-enabled simulation platform

Show Author's Information Zheng Xu,  Yihai Fang,  Nan Zheng,  Hai L. Vu
Department of Civil Engineering, Monash University, Clayton, Australia
Keywords:
Virtual reality, Autonomous vehicle, Mixed traffic, Naturalistic simulation, Cognitive bias, Intervention behavior
Cite this article:
Xu Z, Fang Y, Zheng N, et al. Analyzing the inconsistency in driving patterns between manual and autonomous modes under complex driving scenarios with a VR-enabled simulation platform. Journal of Intelligent and Connected Vehicles, 2022, 5(3): 215-234. https://doi.org/10.1108/JICV-05-2022-0017
Download citation
EndNote(RIS)
BibTeX

539

Views

28

Downloads

Citations

12

Crossref

N/A

WoS

11

Scopus

N/A

CSCD

Abstract Full text About this article
Purpose

With the aid of naturalistic simulations, this paper aims to investigate human behavior during manual and autonomous driving modes in complex scenarios.

Design/methodology/approach

The simulation environment is established by integrating virtual reality interface with a micro-simulation model. In the simulation, the vehicle autonomy is developed by a framework that integrates artificial neural networks and genetic algorithms. Humansubject experiments are carried, and participants are asked to virtually sit in the developed autonomous vehicle (AV) that allows for both human driving and autopilot functions within a mixed traffic environment.

Findings

Not surprisingly, the inconsistency is identified between two driving modes, in which the AV's driving maneuver causes the cognitive bias and makes participants feel unsafe. Even though only a shallow portion of the cases that the AV ended up with an accident during the testing stage, participants still frequently intervened during the AV operation. On a similar note, even though the statistical results reflect that the AV drives under perceived high-risk conditions, rarely an actual crash can happen. This suggests that the classic safety surrogate measurement, e.g. time-tocollision, may require adjustment for the mixed traffic flow.

Research limitations/implications

Understanding the behavior of AVs and the behavioral difference between AVs and human drivers are important, where the developed platform is only the first effort to identify the critical scenarios where the AVs might fail to react.

Practical implications

This paper attempts to fill the existing research gap in preparing close-to-reality tools for AV experience and further understanding human behavior during high-level autonomous driving.

Social implications

This work aims to systematically analyze the inconsistency in driving patterns between manual and autopilot modes in various driving scenarios (i.e. multiple scenes and various traffic conditions) to facilitate user acceptance of AV technology.

Originality/value

A close-to-reality tool for AV experience and AV-related behavioral study. A systematic analysis in relation to the inconsistency in driving patterns between manual and autonomous driving. A foundation for identifying the critical scenarios where the AVs might fail to react.


menu
Abstract
Full text
Outline
About this article

Analyzing the inconsistency in driving patterns between manual and autonomous modes under complex driving scenarios with a VR-enabled simulation platform

Show Author's information Zheng Xu, Yihai Fang, Nan Zheng, Hai L. Vu
Department of Civil Engineering, Monash University, Clayton, Australia

Abstract

Purpose

With the aid of naturalistic simulations, this paper aims to investigate human behavior during manual and autonomous driving modes in complex scenarios.

Design/methodology/approach

The simulation environment is established by integrating virtual reality interface with a micro-simulation model. In the simulation, the vehicle autonomy is developed by a framework that integrates artificial neural networks and genetic algorithms. Humansubject experiments are carried, and participants are asked to virtually sit in the developed autonomous vehicle (AV) that allows for both human driving and autopilot functions within a mixed traffic environment.

Findings

Not surprisingly, the inconsistency is identified between two driving modes, in which the AV's driving maneuver causes the cognitive bias and makes participants feel unsafe. Even though only a shallow portion of the cases that the AV ended up with an accident during the testing stage, participants still frequently intervened during the AV operation. On a similar note, even though the statistical results reflect that the AV drives under perceived high-risk conditions, rarely an actual crash can happen. This suggests that the classic safety surrogate measurement, e.g. time-tocollision, may require adjustment for the mixed traffic flow.

Research limitations/implications

Understanding the behavior of AVs and the behavioral difference between AVs and human drivers are important, where the developed platform is only the first effort to identify the critical scenarios where the AVs might fail to react.

Practical implications

This paper attempts to fill the existing research gap in preparing close-to-reality tools for AV experience and further understanding human behavior during high-level autonomous driving.

Social implications

This work aims to systematically analyze the inconsistency in driving patterns between manual and autopilot modes in various driving scenarios (i.e. multiple scenes and various traffic conditions) to facilitate user acceptance of AV technology.

Originality/value

A close-to-reality tool for AV experience and AV-related behavioral study. A systematic analysis in relation to the inconsistency in driving patterns between manual and autonomous driving. A foundation for identifying the critical scenarios where the AVs might fail to react.

Keywords: Virtual reality, Autonomous vehicle, Mixed traffic, Naturalistic simulation, Cognitive bias, Intervention behavior

References(72)

Adnan, N., Nordin, S.M., bin Bahruddin, M.A. and Ali, M. (2018), “How trust can drive forward the user acceptance to the technology? In-vehicle technology for autonomous vehicle”, Transportation Research Part A: Policy and Practice, Vol. 118, pp. 819-836.
DOI Google Scholar

Bonnefon, J.F., Shariff, A. and Rahwan, I. (2016), “The social dilemma of autonomous vehicles”, Science, Vol. 352 No. 6293, pp. 1573-1576.
DOI Google Scholar
Brown, B., Park, D., Sheehan, B., Shikoff, S., Solomon, J., Yang, J. and Kim, I. (2018), “Assessment of human driver safety at dilemma zones with automated vehicles through a virtual reality environment”, 2018 Systems and Information Engineering Design Symposium (SIEDS), IEEE, pp. 185-190.
Bureau of Infrastructure and Transport Research Economics (BITRE) (2022), “Road trauma Australia 2021 statistical summary”, BITRE, Canberra ACT.
Campbell, S., O'Mahony, N., Krpalcova, L., Riordan, D., Walsh, J., Murphy, A. and Ryan, C. (2018), “Sensor technology in autonomous vehicles: a review”, 2018 29th Irish Signals and Systems Conference (ISSC), IEEE, pp. 1-4.

Choi, J.K. and Ji, Y.G. (2015), “Investigating the importance of trust on adopting an autonomous vehicle”, International Journal of Human-Computer Interaction, Vol. 31 No. 10, pp. 692-702.
DOI Google Scholar
Da Silva, IN., Spatti, D.H., Flauzino, R.A., Liboni, L.H.B. and dos Reis Alves, S.F. (2017), “Artificial neural network architectures and training processes”, Artificial Neural Networks, Springer.

Daamen, W., Loot, M. and Hoogendoorn, S.P. (2010), “Empirical analysis of merging behavior at freeway on-ramp”, Transportation Research Record: Journal of the Transportation Research Board, Vol. 2188 No. 1, pp. 108-118.
DOI Google Scholar

Detjen, H., Faltaous, S., Pfleging, B., Geisler, S. and Schneegass, S. (2021), “How to increase automated vehicles’ acceptance through in-vehicle interaction design: a review”, International Journal of Human–Computer Interaction, Vol. 37 No. 4, pp. 308-330.
DOI Google Scholar
Discover. data. vic. gov. au (2021), “Typical hourly traffic volume – victorian government data directory”, available at: https://discover.data.vic.gov.au/dataset/typical-daily-traffic-volume-profile (accessed 15 December 2021).
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A. and Koltun, V. (2017), “CARLA: an open urban driving simulator”, Conference on robot learning, PMLR, pp. 1-16.

Duarte, F. and Ratti, C. (2018), “The impact of autonomous vehicles on cities: a review”, Journal of Urban Technology, Vol. 25 No. 4, pp. 3-18.
DOI Google Scholar
Ejercito, P.M., Nebrija, K.G.E., Feria, R.P. and Lara-Figueroa, L.L. (2017), “Traffic simulation software review”, 2017 8th International Conference on Information, Intelligence, Systems & Applications (IISA), IEEE, pp. 1-4.

Elmquist, A. and Negrut, D. (2020), “Methods and models for simulating autonomous vehicle sensors”, IEEE Transactions on Intelligent Vehicles, Vol. 5 No. 4, pp. 684-692.
DOI Google Scholar

Fang, J., Zhou, D., Yan, F., Zhao, T., Zhang, F., Ma, Y., Wang, L. and Yang, R. (2020), “Augmented LiDAR simulator for autonomous driving”, IEEE Robotics and Automation Letters, Vol. 5 No. 2, pp. 1931-1938.
DOI Google Scholar

Feng, S., Yan, X., Sun, H., Feng, Y. and Liu, H.X. (2021), “Intelligent driving intelligence test for autonomous vehicles with naturalistic and adversarial environment”, Nature Communications, Vol. 12, pp. 1-14.
DOI Google Scholar
Gao, S., Paulissen, S., Coletti, M. and Patton, R. (2021), “Quantitative evaluation of autonomous driving in CARLA”, 2021 IEEE Intelligent Vehicles Symposium Workshops (IV Workshops), IEEE, pp. 257-263.

Gettman, D. and Head, L. (2003), “Surrogate safety measures from traffic simulation models”, Transportation Research Record: Journal of the Transportation Research Board, Vol. 1840 No. 1, pp. 104-115.
DOI Google Scholar
Goedicke, D., Li, J., Evers, V. and Ju, W. (2018), “Vr-oom: virtual reality on-road driving simulation”, Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1-11.
Goh, J., Hu, S. and Fang, Y. (2019), “Human-in-the-loop simulation for crane lift planning in modular construction on-site assembly”, Computing in Civil Engineering 2019: Visualization, Information Modeling, and Simulation, American Society of Civil Engineers, Reston, VA, pp. 71-78.
Gu, T. and Dolan, J.M. (2012), “On-road motion planning for autonomous vehicles”, International Conference on Intelligent Robotics and Applications, Springer, pp. 588-597.
Haarnoja, T., Zhou, A., Abbeel, P. and Levine, S. (2018), “Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor”, International conference on machine learning, PMLR, pp. 1861-1870.
Henne, M., Schwaiger, A. and Weiss, G. (2019), “Managing uncertainty of AI-based perception for autonomous systems”, AISafety@ IJCAI.

Hoogendoorn, R., van Arerm, B. and Hoogendoom, S. (2014), “Automated driving, traffic flow efficiency, and human factors: literature review”, Transportation Research Record: Journal of the Transportation Research Board, Vol. 2422 No. 1, pp. 113-120.
DOI Google Scholar
Huang, W., Wang, K., Lv, Y. and Zhu, F. (2016), “Autonomous vehicles testing methods review”, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), IEEE, pp. 163-168.

Hussein, A., Gaber, M.M., Elyan, E. and Jayne, C. (2017), “Imitation learning: a survey of learning methods”, ACM Computing Surveys, Vol. 50 No. 2, pp. 1-35.
DOI Google Scholar

Jing, P., Xu, G., Chen, Y., Shi, Y. and Zhan, F. (2020), “The determinants behind the acceptance of autonomous vehicles: a systematic review”, Sustainability, Vol. 12 No. 5, p. 1719.
DOI Google Scholar
Juliani, A., Berges, V.P., Teng, E., Cohen, A., Harper, J., Elion, C., Goy, C., Gao, Y., Henry, H., Mattar, M. and Lange, D. (2018), “Unity: a general platform for intelligent agents”, arXiv preprint arXiv: 1809.02627.

Kalra, N. and Paddock, S.M. (2016), “Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability?”, Transportation Research Part A: Policy and Practice, Vol. 94, pp. 182-193.
DOI Google Scholar

Kennedy, R.S., Lane, N.E., Berbaum, K.S. and Lilienthal, M.G. (1993), “Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness”, The International Journal of Aviation Psychology, Vol. 3 No. 3, pp. 203-220.
DOI Google Scholar
Kocić, J., Jovičić, N. and Drndarević, V. (2018), “Sensors and sensor fusion in autonomous vehicles”, 2018 26th Telecommunications Forum (TELFOR), IEEE, pp. 420-425.
Kosecka, J., Blasi, R., Taylor, C.J. and Malik, J. (1997), “Vision-based lateral control of vehicles”, Proceedings of Conference on Intelligent Transportation Systems, IEEE, pp. 900-905.

Kriegeskorte, N. and Golan, T. (2019), “Neural network models and deep learning”, Current Biology, Vol. 29 No. 7, pp. R231-R236.
DOI Google Scholar
Kuribayashi, A., Takeuchi, E., Carballo, A., Ishiguro, Y. and Takeda, K. (2021), “A recognition phase intervention interface to improve naturalness of autonomous driving for distracted drivers”, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC), IEEE, pp. 1737-1744.

Kuutti, S., Bowden, R., Jin, Y., Barber, P. and Fallah, S. (2020), “A survey of deep learning applications to autonomous vehicle control”, IEEE Transactions on Intelligent Transportation Systems, Vol. 22 No. 2, pp. 712-733.
Google Scholar

Lee, D.N. (1976), “A theory of visual control of braking based on information about time-to-collision”, Perception, Vol. 5 No. 4, pp. 437-459.
DOI Google Scholar

Li, W., Pan, C., Zhang, R., Ren, J., Ma, Y., Fang, J., Yan, F., Geng, Q., Huang, X. and Gong, H. (2019), “AADS: augmented autonomous driving simulation using data-driven algorithms”, Science Robotics, Vol. 4 No. 28, p. eaaw0863.
DOI Google Scholar

Lim, K.L., Whitehead, J., Jia, D. and Zheng, Z. (2021), “State of data platforms for connected vehicles and infrastructures”, Communications in Transportation Research, Vol. 1, p. 100013.
DOI Google Scholar
Liu, S. (2020), Engineering Autonomous Vehicles and Robots: The DragonFly Modular-Based Approach, John Wiley & Sons.

Morando, M.M., Tian, Q., Truong, L.T. and Vu, H.L. (2018), “Studying the safety impact of autonomous vehicles using simulation-based surrogate safety measures”, Journal of Advanced Transportation, Vol. 2018.
Google Scholar

Morra, L., Lamberti, F., Pratticó, F.G., La Rosa, S. and Montuschi, P. (2019), “Building trust in autonomous vehicles: role of virtual reality driving simulators in HMI design”, IEEE Transactions on Vehicular Technology, Vol. 68 No. 10, pp. 9438-9450.
DOI Google Scholar
Nunnally, J. and Bernstein, I. (1994), “The assessment of reliability”, Psychometric Theory, 3rd ed., McGraw-Hill, New York, NY, pp. 3, 248-292.
NVIDIA (2017), “NVIDIA DRIVE CONSTELLATION: virtual reality autonomous vehicle simulator”, available at: https://developer.nvidia.com/drive/drive-constellation (accessed 10 December 2021).

Peng, B., Keskin, M.F., Kulcsár, B. and Wymeersch, H. (2021), “Connected autonomous vehicles for improving mixed traffic efficiency in unsignalized intersections with deep reinforcement learning”, Communications in Transportation Research, Vol. 1, p. 100017.
DOI Google Scholar

Pérez-Gil, O., Barea, R., López-Guillén, E., Bergasa, L.M., Gómez-Huelamo, C., Gutiérrez, R. and Dí az-Dí az, A. (2022), “Deep reinforcement learning based control for autonomous vehicles in CARLA”, Multimedia Tools and Applications, Vol. 81 No. 3, pp. 1-24.
Google Scholar

Ramadhan, S.A., Joelianto, E. and Sutarto, H.Y. (2019), “Simulation of traffic control using Vissim-COM interface”, Internetworking Indonesia Journal, Vol. 11 No. 1, pp. 55-61.
Google Scholar
Reza, M., Choudhury, S., Dash, J.K. and Roy, D.S. (2020), “An ai-based real-time roadway-environment perception for autonomous driving”, 2020 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan), IEEE, pp. 1-2.
Rong, G., Shin, B.H., Tabatabaee, H., Lu, Q., Lemke, S., Možeiko, M., Boise, E., Uhm, G., Gerow, M. and Mehta, S. (2020), “Lgsvl simulator: a high fidelity simulator for autonomous driving”, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), IEEE, pp. 1-6.

Rosique, F., Navarro, P.J., Fernández, C. and Padilla, A. (2019), “A systematic review of perception system and simulators for autonomous vehicles research”, Sensors, Vol. 19 No. 3, p. 648.
DOI Google Scholar
SAE International (2018), “Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles (J3016_201806)”, available at: www.sae.org/standards/content/j3016_201806/ (accessed 1 December 2021).
Schulman, J., Wolski, F., Dhariwal, P., Radford, A. and Klimov, O. (2017), “Proximal policy optimization algorithms”, arXiv preprint arXiv: 1707.06347.

Shah, S., Dey, D., Lovett, C. and Kapoor, A. (2018), “Airsim: high-fidelity visual and physical simulation for autonomous vehicles”, Field and Service Robotics, Springer, pp. 621-635.
Google Scholar

Soteropoulos, A., Berger, M. and Ciari, F. (2019), “Impacts of automated vehicles on travel behaviour and land use: an international review of modelling studies”, Transport Reviews, Vol. 39 No. 1, pp. 29-49.
DOI Google Scholar

Sportillo, D., Paljic, A. and Ojeda, L. (2018), “Get ready for automated driving using virtual reality”, Accident Analysis & Prevention, Vol. 118, pp. 102-113.
Google Scholar
Thorn, A. (2020), “3D lighting and materials”, Moving from Unity to Godot, Apress, Berkeley, CA, pp. 161-200.

Uteshev, A.Y. and Goncharova, M.V. (2018), “Point-to-ellipse and point-to-ellipsoid distance equation analysis”, Journal of Computational and Applied Mathematics, Vol. 328, pp. 232-251.
DOI Google Scholar

Van Brummelen, J., O’Brien, M., Gruyer, D. and Najjaran, H. (2018), “Autonomous vehicle perception: the technology of today and tomorrow”, Transportation Research Part C: Emerging Technologies, Vol. 89, pp. 384-406.
DOI Google Scholar
Van Der Horst, R. and Hogema, J. (1993), “Time-to-collision and collision avoidance systems”, Proceedings of the 6th ICTCT Workshop.

Van Der Laan, J.D., Heino, A. and De Waard, D. (1997), “A simple procedure for the assessment of acceptance of advanced transport telematics”, Transportation Research Part C: Emerging Technologies, Vol. 5 No. 1, pp. 1-10.
DOI Google Scholar
Vicroads (2022), “Coordinated ramp signals”, available at: www.vicroads.vic.gov.au/traffic-and-road-use/traffic-management/managed-motorways/coordinated-ramp-signals (accessed 14 December 2021).
Vicroads. vic. gov. au (2021), “Crash statistics: VicRoads”, available at: www.vicroads.vic.gov.au/safety-and-road-rules/safety-statistics/crash-statistics (accessed 15 December 2021).

Wang, Z., Fang, J., Dai, X., Zhang, H. and Vlacic, L. (2020), “Intelligent vehicle self-localization based on double-layer features and multilayer LIDAR”, IEEE Transactions on Intelligent Vehicles, Vol. 5 No. 4, pp. 616-625.
DOI Google Scholar
WHO, V. (2018), “Global status report on road safety 2018”, World Health Organization.

Wilson, K.M., Yang, S., Roady, T., Kuo, J. and Lenné, M.G. (2020), “Driver trust & mode confusion in an on-road study of level-2 automated vehicle technology”, Safety Science, Vol. 130, p. 104845.
DOI Google Scholar

Wynne, R.A., Beanland, V. and Salmon, P.M. (2019), “Systematic review of driving simulator validation studies”, Safety Science, Vol. 117, pp. 138-151.
DOI Google Scholar
Xiong, X. Wang, J. Zhang, F. and Li, K. (2016), “Combining deep reinforcement learning and safety based control for autonomous driving”, arXiv preprint arXiv: 1612.00147.

Xu, Z. and Zheng, N. (2021), “Incorporating virtual reality technology in safety training solution for construction site of urban cities”, Sustainability, Vol. 13 No. 1, p. 243.
Google Scholar

Xu, Z., Zou, X., Oh, T. and Vu, H.L. (2021), “Studying freeway merging conflicts using virtual reality technology”, Journal of Safety Research, Vol. 76, pp. 16-29.
DOI Google Scholar
Yousfi, E., Malin, S., Halit, L., Roger, S. and Dogan, E. (2021), “Driver experience and safety during manual intervention in a simulated automated vehicle: influence of longer time margin allowed by connectivity”, European Conference on Cognitive Ergonomics 2021, pp. 1-7.

Yu, B., Bao, S., Zhang, Y., Sullivan, J. and Flannagan, M. (2021), “Measurement and prediction of driver trust in automated vehicle technologies: an application of hand position transition probability matrix”, Transportation Research Part C: Emerging Technologies, Vol. 124, p. 102957.
DOI Google Scholar

Zheng, L., Sayed, T. and Mannering, F. (2021), “Modeling traffic conflicts for use in road safety analysis: a review of analytic methods and future directions”, Analytic Methods in Accident Research, Vol. 29, p. 100142.
DOI Google Scholar

Zou, X., O'Hern, S., Ens, B., Coxon, S., Mater, P., Chow, R., Neylan, M. and Vu, H.L. (2021), “On-road virtual reality autonomous vehicle (VRAV) simulator: an empirical study on user experience”, Transportation Research Part C: Emerging Technologies, Vol. 126, p. 103090.
DOI Google Scholar
Publication history
Copyright
Rights and permissions

Publication history

Received: 03 May 2022
Revised: 17 June 2022
Accepted: 19 June 2022
Published: 12 July 2022
Issue date: October 2022

Copyright

© 2022 Zheng Xu, Yihai Fang, Nan Zheng and Hai L. Vu. Published in Journal of Intelligent and Connected Vehicles. Published by Emerald Publishing Limited.

Rights and permissions

This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence maybe seen at http://creativecommons.org/licences/by/4.0/legalcode

Return