Open Access
Issue
Published:
18 October 2018
Markov probabilistic decision making of self-driving cars in highway with random traffic flow: a simulation study
),
Download citation
395
Views
15
Downloads
23
Crossref
N/A
WoS
23
Scopus
N/A
CSCD
Decision-making is one of the key technologies for self-driving cars. The high dependency of previously existing methods on human driving data or rules makes it difficult to model policies for different driving situations.
In this research, a probabilistic decision-making method based on the Markov decision process (MDP) is proposed to deduce the optimal maneuver automatically in a two-lane highway scenario without using any human data. The decision-making issues in a traffic environment are formulated as the MDP by defining basic elements including states, actions and basic models. Transition and reward models are defined by using a complete prediction model of the surrounding cars. An optimal policy was deduced using a dynamic programing method and evaluated under a two-dimensional simulation environment.
Results show that, at the given scenario, the self-driving car maintained safety and efficiency with the proposed policy.
This paper presents a framework used to derive a driving policy for self-driving cars without relying on any human driving data or rules modeled by hand.
menu
Markov probabilistic decision making of self-driving cars in highway with random traffic flow: a simulation study
),
Abstract
Decision-making is one of the key technologies for self-driving cars. The high dependency of previously existing methods on human driving data or rules makes it difficult to model policies for different driving situations.
In this research, a probabilistic decision-making method based on the Markov decision process (MDP) is proposed to deduce the optimal maneuver automatically in a two-lane highway scenario without using any human data. The decision-making issues in a traffic environment are formulated as the MDP by defining basic elements including states, actions and basic models. Transition and reward models are defined by using a complete prediction model of the surrounding cars. An optimal policy was deduced using a dynamic programing method and evaluated under a two-dimensional simulation environment.
Results show that, at the given scenario, the self-driving car maintained safety and efficiency with the proposed policy.
This paper presents a framework used to derive a driving policy for self-driving cars without relying on any human driving data or rules modeled by hand.
References(19)
Bacha, A., Bauman, C., Faruque, R., Fleming, M., Terwelp, C., Reinholtz, C., Hong, D., Wicks, A., Alberi, T., Anderson, D. and Cacciola, S. (2008), “Odin: team VictorTango’s entry in the DARPA Urban Challenge”, Journal of Field Robotics, Vol. 25 No. 8, pp. 467-492.
Bojarski, M., Del Testa, D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J. and Zhang, X. (2016), “End to end learning for self-driving cars”, arXiv Preprint arXiv, Vol. 1604, p. 07316.
Bojarski, M., Yeres, P., Choromanska, A., Choromanski, K., Firner, B., Jackel, L. and Muller, U. (2017), “Explaining how a deep neural network trained with end-to-end learning steers a car”, arXiv Preprint arXiv, Vol. 1704, p. 7911.
Buehler, M., Iagnemma, K., and Singh, S. (2009), The DARPA Urban Challenge: Autonomous Vehicles in City Traffic, Springer, Berlin, Vol. 56.
Carvalho, A., Lefévre, S., Schildbach, G., Kong, J. and Borrelli, F. (2015), “Automated driving: the role of forecasts and uncertainty a control perspective”, European Journal of Control, Vol. 24, pp. 14-32.
Furda, A. and Vlacic, L. (2011), “Enabling safe autonomous driving in real-world city traffic using multiple criteria decision making”, IEEE Intelligent Transportation Systems Magazine, Vol. 3 No. 1, pp. 4-17.
Janai, J., Gü ney, F., Behl, A. and Geiger, A. (2017), “Computer vision for autonomous vehicles: problems, datasets and state-of-the-art”, arXiv Preprint arXiv, Vol. 1704, p. 05519.
Lefèvre, S., Vasquez, D. and Laugier, C. (2014), “A survey on motion prediction and risk assessment for intelligent vehicles”, Robomech Journal, Vol. 1 No. 1, p. 1.
Li, S.E., Qin, X., Li, K., Wang, J. and Xie, B. (2017b), “Robustness analysis and controller synthesis of homogeneous vehicular platoons with bounded parameter uncertainty”, IEEEASME Transactions on Mechatronics, Vol. 22 No. 2, pp. 1014-1025.
Li, S.E., Zheng, Y., Li, K., Wu, Y., Hedrick, J.K., Gao, F. and Zhang, H. (2017c), “Dynamical modeling and distributed control of connected and automated vehicles: challenges and opportunities”, IEEE Intelligent Transportation Systems Magazine, Vol. 9 No. 3, pp. 46-58.
Liu, C., Li, S.E., Yang, D. and Hedrick, J.K. (2018), “Distributed Bayesian filter using measurement dissemination for multiple unmanned ground vehicles with dynamically changing interaction topologies”, Journal of Dynamic Systems, Measurement, and Control, Vol. 140 No. 3, p. 030903.
Montemerlo, M., Becker, J., Bhat, S., Dahlkamp, H., Dolgov, D., Ettinger, S., Haehnel, D., Hilden, T., Hoffmann, G., Huhnke, B., et al. (2008), “Junior: the Stanford entry in the urban challenge”, Journal of Field Robotics, Vol. 25 No. 9, pp. 569-597.
Paden, B., Čáp, M., Yong, S.Z., Yershov, D. and Frazzoli, E. (2016), “A survey of motion planning and control techniques for self-driving urban vehicles”, IEEE Transactions on Intelligent Vehicles, Vol. 1 No. 1, pp. 33-55.
Patole, S.M., Torlak, M., Wang, D. and Ali, M. (2017), “Automotive radars: a review of signal processing techniques”, IEEE Signal Processing Magazine, Vol. 34 No. 2, pp. 22-35.
Urmson, C., Anhalt, J., Bagnell, D., Baker, C., Bittner, R., Clark, M., Dolan, J., Duggins, D., Galatali, T., Geyer, C. and Gittleman, M. (2008), “Autonomous driving in urban environments: boss and the urban challenge”, Journal of Field Robotics, Vol. 25 No. 8, pp. 425-466.
Zheng, Y., Li, S.E., Li, K., Borrelli, F. and Hedrick, J.K. (2017), “Distributed model predictive control for heterogeneous vehicle platoons under unidirectional topologies”, IEEE Transactions on Control Systems Technology, Vol. 25 No. 3, pp. 899-910.
Publication history
Copyright
Rights and permissions
This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
FEEDBACK 
TOP