AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Communications in Transportation Research Article
Article Link
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

Towards explainable traffic flow prediction with large language models

Xusen Guoa,1, Qiming Zhanga,1, Junyue Jiangb, Mingxing Penga, Meixin Zhua,c( ), Hao Frank Yangb( )
Intelligent Transportation Thrust, System Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 511400, China
Department of Civil and System Engineering, Johns Hopkins University, Baltimore, 21218, USA
Guangdong Provincial Key Lab of Integrated Communication, Sensing and Computation for Ubiquitous Internet of Things, Guangzhou, 511400, China

1 Xusen Guo and Qiming Zhang contributed equally to this work.

Show Author Information

Highlights

• Multi-modality traffic forecasting dataset for the learning-based prediction tasks.

• Traffic flow prediction with large language models, accountable and reliable prediction results.

• Spatial-temporal alignment, zero-shot learning capability to other unseen traffic prediction tasks.

Abstract

Traffic forecasting is crucial for intelligent transportation systems. It has experienced significant advancements thanks to the power of deep learning in capturing latent patterns of traffic data. However, recent deep-learning architectures require intricate model designs and lack an intuitive understanding of the mapping from input data to predicted results. Achieving both accuracy and explainability in traffic prediction models remains a challenge due to the complexity of traffic data and the inherent opacity of deep learning models. To tackle these challenges, we propose a traffic flow prediction model based on large language models (LLMs) to generate explainable traffic predictions, named xTP-LLM. By transferring multi-modal traffic data into natural language descriptions, xTP-LLM captures complex time-series patterns and external factors from comprehensive traffic data. The LLM framework is fine-tuned using language-based instructions to align with spatial-temporal traffic flow data. Empirically, xTP-LLM shows competitive accuracy compared with deep learning baselines, while providing an intuitive and reliable explanation for predictions. This study contributes to advancing explainable traffic prediction models and lays a foundation for future exploration of LLM applications in transportation.

Keywords

Traffic flow predictionLarge language modelsSpatial-temporal predictionExplainability

References

 

Asanjan, A., Yang, T., Hsu, K., Sorooshian, S., Lin, J., Peng, Q., 2018. Short-term precipitation forecast based on the PERSIANN system and LSTM recurrent neural networks. J. Geophys. Res. Atmos. 123, 12–543.
Crossref Google Scholar
 
Ates, E., Aksar, B., Leung, V.J., Coskun, A.K., 2021. Counterfactual explanations for multivariate time series. In: 2021 International Conference on Applied Artificial Intelligence, pp. 1–8.
Crossref
 
Bai, L., Yao, L., Li, C., Wang, X., Wang, C., 2020. Adaptive graph convolutional recurrent network for traffic forecasting. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, pp. 17804–17815.
 
Barredo-Arrieta, A., Laña, I., Del Ser, J., 2019. What lies beneath: a note on the explainability of black-box machine learning models for road traffic forecasting. In: 2019 IEEE Intelligent Transportation Systems Conference. IEEE, pp. 2232–2237.
Crossref
 

Cai, L., Janowicz, K., Mai, G., Yan, B., Zhu, R., 2020. Traffic transformer: capturing the continuity and periodicity of time series for traffic forecasting. Transactions in GIS 24, 736–755.
Crossref Google Scholar
 
Cambria, E., Malandri, L., Mercorio, F., Nobani, N., Seveso, A., 2024. XAI meets LLMs: a survey of the relation between explainable AI and large language models. https://doi.org/10.48550/arXiv.2407.15248.
 
Chang, E.Y., 2023. Examining GPT-4’s capabilities and enhancement with socrasynth. In: The 10th International Conference on Computational Science and Computational Intelligence, pp. 7–14.
Crossref
 

Chen, Y., Chen, H., Ye, P., Lv, Y., Wang, F.Y., 2022. Acting as a decision maker: traffic-condition-aware ensemble learning for traffic flow prediction. IEEE Trans. Intell. Transport. Syst. 23, 3190–3200.
Crossref Google Scholar
 
Chen, Y., Wang, X., Xu, G., 2023. GATGPT: a pre-trained large language model with graphattention network for spatiotemporal imputation. https://doi.org/10.48550/arXiv.2311.14332.
 

Dai, L., Wang, L., 2020. Nonlinear analysis of high accuracy and reliability in traffic flow prediction. Nonlinear Eng. 9, 290–298.
Crossref Google Scholar
 

Dimitrakopoulos, G., Demestichas, P., 2010. Intelligent transportation systems. IEEE Veh. Technol. Mag. 5, 77–84.
Crossref Google Scholar
 

Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., et al., 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nat. Mach. Intell. 5, 220–235.
Crossref Google Scholar
 

Du, S., Li, T., Gong, X., Horng, S.J., 2020. A hybrid method for traffic flow forecasting using multimodal deep learning. Int. J. Comput. Intell. Syst. 13, 85–97.
Crossref Google Scholar
 
Fan, Z., Song, X., Xia, T., Jiang, R., Shibasaki, R., Sakuramachi, R., 2018. Online deep ensemble learning for predicting citywide human mobility. In: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, pp. 1–21.
Crossref
 
Fang, Z., Long, Q., Song, G., Xie, K., 2021. Spatial-temporal graph ODE networks for traffic flow forecasting. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 364–373.
Crossref
 
Gao, Q., Trajcevski, G., Zhou, F., Zhang, K., Zhong, T., Zhang, F., 2018. Trajectory-based social circle inference. In: Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 369–378.
Crossref
 
Gruver, N., Finzi, M., Qiu, S., Wilson, A.G., 2024. Large language models are zero-shot time series forecasters. In: Proceedings of the 37th International Conference on Neural Information Processing Systems, pp. 19622–19635.
 
Guo, S., Lin, Y., Feng, N., Song, C., Wan, H., 2019. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 922–929.
Crossref
 
Hochreiter, S., 1997. Long short-term memory. Neural Computation MIT-Press 9, 1735–1780.
Crossref
 

Hopfield, J.J., 1982. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558.
Crossref Google Scholar
 
Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., et al., 2021. Lora: low-rank adaptation of large language models. https://doi.org/10.48550/arXiv.2106.09685.
 
Huang, S., Mamidanna, S., Jangam, S., Zhou, Y., Gilpin, L.H., 2023. Can large language models explain themselves? A study of LLM-generated self-explanations. https://doi.org/10.48550/arXiv.2310.11207.
 
Jiang, J., Han, C., Zhao, W.X., Wang, J., 2023. PDFormer: Propagation delay-aware dynamic long-range transformer for traffic flow prediction, pp. 4365–4373.
Crossref
 
Jin, M., Wang, S., Ma, L., Chu, Z., Zhang, J.Y., Shi, X., et al., 2023. Time-LLM: time series forecasting by reprogramming large language models. https://doi.org/10.4855 0/arXiv.2310.01728.
 
Jin, W., Lin, Y., Wu, Z., Wan, H., 2018. Spatio-temporal recurrent convolutional networks for citywide short-term crowd flows prediction. In: Proceedings of the 2nd International Conference on Compute and Data Analysis, pp. 28–35.
Crossref
 
Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y., 2022. Large language models are zero-shot reasoners. In: Proceedings of the 36th International Conference on Neural Information Processing Systems, pp. 22199–22213.
 
Lan, S., Ma, Y., Huang, W., Wang, W., Yang, H., Li, P., 2022. Dstagnn: dynamic spatialtemporal aware graph neural network for traffic flow forecasting. In: International Conference on Machine Learning, pp. 11906–11917.
 
Li, Y., Yu, R., Shahabi, C., Liu, Y., 2017. Diffusion convolutional recurrent neural network: data-driven traffic forecasting. https://doi.org/10.48550/arXiv.1707.01926.
 
Liang, Y., Ke, S., Zhang, J., Yi, X., Zheng, Y., 2018. Geoman: multi-level attention networks for geo-sensory time series prediction. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 3428–3434.
Crossref
 
Liu, S., Cheng, H., Liu, H., Zhang, H., Li, F., Ren, T., et al., 2023. Llava-plus: learning to use tools for creating multimodal agents. https://doi.org/10.48550/arXiv.2311.05437.
Crossref
 
Liu, X., Xia, Y., Liang, Y., Hu, J., Wang, Y., Bai, L., et al., 2024. LargeST: a benchmark dataset for large-scale traffic forecasting. In: Proceedings of the 37th International Conference on Neural Information Processing Systems, pp. 75354–75371.
 

Marblestone, A.H., Wayne, G., Kording, K.P., 2016. Toward an integration of deep learning and neuroscience. Front. Comput. Neurosci. 10, 2159–42202.
Crossref Google Scholar
 
Peng, M., Guo, X., Chen, X., Zhu, M., Chen, K., Wang, X., et al., 2024. LC-LLM: explainable lane-change intention and trajectory predictions with large language models. https://doi.org/10.48550/arXiv.2403.18344.
 

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al., 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 1–9.
Google Scholar
 

Ranjan, N., Bhandari, S., Zhao, H.P., Kim, H., Khan, P., 2020. City-wide traffic congestion prediction based on CNN, LSTM and transpose CNN. IEEE Access 8, 81606–81620.
Crossref Google Scholar
 

Ray, P.P., 2023. ChatGPT: a comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems 3, 121–154.
Crossref Google Scholar
 
Shi, X., Xue, S., Wang, K., Zhou, F., Zhang, J., Zhou, J., et al., 2023. Language models can improve event prediction by few-shot abductive reasoning. In: Proceedings of the 37th International Conference on Neural Information Processing Systems, pp. 29532–29557.
 

Thirunavukarasu, A.J., Ting, D.S.J., Elangovan, K., Gutierrez, L., Tan, T.F., Ting, D.S.W., 2023. Large language models in medicine. Nat. Med. 29, 1930–1940.
Crossref Google Scholar
 
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., et al., 2023. Llama 2: open foundation and fine-tuned chat models. https://doi.org/10.48550/arXiv.2307.09288.
 
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., et al., 2017. Attention is all you need. In: Proceedings of the 31th International Conference on Neural Information Processing Systems, pp. 6000–6010.
 
Wan, H., Feng, S., Tan, Z., Wang, H., Tsvetkov, Y., Luo, M., 2024. DELL: generating reactions and explanations for LLM-based misinformation detection. In: Findings of the Association for Computational Linguistics ACL 2024, pp. 2637–2667.
Crossref
 

Wang, S., Cao, J., Yu, P.S., 2022. Deep learning for spatio-temporal data mining: a survey. IEEE Trans. Knowl. Data Eng. 34, 3681–3700.
Crossref Google Scholar
 
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., et al., 2022. Chain-ofthought prompting elicits reasoning in large language models. In: Proceedings of the 36th International Conference on Neural Information Processing Systems, pp. 24824–24837.
 
Wu, S., Irsoy, O., Lu, S., Dabravolski, V., Dredze, M., Gehrmann, S., et al., 2023. BloombergGPT: a large language model for finance. https://doi.org/10.48550/arXiv.2303.17564.
 
Wu, Z., Pan, S., Long, G., Jiang, J., Zhang, C., 2019. Graph wavenet for deep spatialtemporal graph modeling. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 1907–1913.
Crossref
 
Xu, M., Dai, W., Liu, C., Gao, X., Lin, W., Qi, G.J., et al., 2020. Spatial-temporal transformer networks for traffic flow forecasting. https://doi.org/10.48550/arXiv.2001.02908.
 
Yan, J., Wang, H., 2023. Self-interpretable time series prediction with counterfactual explanations. In: Proceedings of the 40th International Conference on Machine Learning, pp. 39110–39125.
 

Yang, H.F., Dillon, T.S., Chen, Y.P.P., 2016. Optimized structure of the traffic flow forecasting model with a deep learning approach. IEEE Transact. Neural Networks Learn. Syst. 28, 2371–2381.
Crossref Google Scholar
 

Yannis, G., Dragomanovits, A., Laiou, A., La Torre, F., Domenichini, L., Richter, T., et al., 2017. Road traffic accident prediction modelling: a literature review. Proceedings of the Institution of Civil Engineers - Transport 170, 245–254.
Crossref Google Scholar
 
Yu, B., Yin, H., Zhu, Z., 2018. Spatio-temporal graph convolutional networks: a deep learning framework for traffic forecasting. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, pp. 3634–3640.
Crossref
 
Yu, C., Ma, X., Ren, J., Zhao, H., Yi, S., 2020. Spatio-temporal graph transformer networks for pedestrian trajectory prediction. In: Proceedings of the 16th European Conference on Computer Vision, pp. 507–523.
Crossref
 
Zhang, J., Zheng, Y., Qi, D., 2017. Deep spatio-temporal residual networks for citywide crowd flows prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 1655–1661.
Crossref
 
Zhang, L., Fu, K., Ji, T., Lu, C.T., 2022. Granger causal inference for interpretable traffic prediction. In: 2022 IEEE 25th International Conference on Intelligent Transportation Systems, pp. 1645–1651.
Crossref
 

Zhou, Y., Li, Y., Zhu, Q., Chen, F., Shao, J., Luo, Y., et al., 2019. A reliable traffic prediction approach for bike-sharing system by exploiting rich information with temporal link prediction strategy. Trans. GIS 23, 1125–1151.
Crossref Google Scholar
 
Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M., 2023. MiniGPT-4: enhancing visionlanguage understanding with advanced large language models. https://doi.org/10.48550/arXiv.2304.10592.
 
Ziegler, D.M., Stiennon, N., Wu, J., Brown, T.B., Radford, A., Amodei, D., et al., 2019. Fine-tuning language models from human preferences. https://doi.org/10.48550/arXiv.1909.08593.
 
Zytek, A., Pido, S., Veeramachaneni, K., 2024. LLMs for XAI: future directions for explaining explanations. https://doi.org/10.48550/arXiv.2405.06064.
Communications in Transportation Research
Volume 4 Issue 4,
December 2024
Article number: 100150
DOI: 10.1016/j.commtr.2024.100150
Cite this article:
Guo X, Zhang Q, Jiang J, et al. Towards explainable traffic flow prediction with large language models. Communications in Transportation Research, 2024, 4(4): 100150. https://doi.org/10.1016/j.commtr.2024.100150

304

Views

3

Crossref

3

Web of Science

4

Scopus

Altmetrics
Received: 23 July 2024
Revised: 29 August 2024
Accepted: 01 September 2024
Published: 02 December 2024
© 2024 The Authors.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Return