Journal Home > Volume 28 , Issue 4

Interactive Recommendation (IR) formulates the recommendation as a multi-step decision-making process which can actively utilize the individuals’ feedback in multiple steps and optimize the long-term user benefit of recommendation. Deep Reinforcement Learning (DRL) has witnessed great application in IR for e-commerce. However, user cold-start problem impairs the learning process of the DRL-based recommendation scheme. Moreover, most existing DRL-based recommendations ignore user relationships or only consider the single-hop social relationships, which cannot fully utilize the social network. The fact that those schemes can not capture the multiple-hop social relationships among users in IR will result in a sub-optimal recommendation. To address the above issues, this paper proposes a Social Graph Neural network-based interactive Recommendation scheme (SGNR), which is a multiple-hop social relationships enhanced DRL framework. Within this framework, the multiple-hop social relationships among users are extracted from the social network via the graph neural network which can sufficiently take advantage of the social network to provide more personalized recommendations and effectively alleviate the user cold-start problem. The experimental results on two real-world datasets demonstrate that the proposed SGNR outperforms other state-of-the-art DRL-based methods that fail to consider social relationships or only consider single-hop social relationships.


menu
Abstract
Full text
Outline
About this article

SGNR: A Social Graph Neural Network Based Interactive Recommendation Scheme for E-Commerce

Show Author's information Dehua Ma1Yufeng Wang1( )Jianhua Ma2Qun Jin3
School of Communications and lnformation Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
Digital Media Department, the Faculty of Computer and Information Sciences, Hosei University, Tokyo 194-0298, Japan
Networked Information Systems Laboratory, Department of Human Informatics and Cognitive Sciences, Faculty of Human Sciences, Waseda University, Tokyo 169-8050, Japan

Abstract

Interactive Recommendation (IR) formulates the recommendation as a multi-step decision-making process which can actively utilize the individuals’ feedback in multiple steps and optimize the long-term user benefit of recommendation. Deep Reinforcement Learning (DRL) has witnessed great application in IR for e-commerce. However, user cold-start problem impairs the learning process of the DRL-based recommendation scheme. Moreover, most existing DRL-based recommendations ignore user relationships or only consider the single-hop social relationships, which cannot fully utilize the social network. The fact that those schemes can not capture the multiple-hop social relationships among users in IR will result in a sub-optimal recommendation. To address the above issues, this paper proposes a Social Graph Neural network-based interactive Recommendation scheme (SGNR), which is a multiple-hop social relationships enhanced DRL framework. Within this framework, the multiple-hop social relationships among users are extracted from the social network via the graph neural network which can sufficiently take advantage of the social network to provide more personalized recommendations and effectively alleviate the user cold-start problem. The experimental results on two real-world datasets demonstrate that the proposed SGNR outperforms other state-of-the-art DRL-based methods that fail to consider social relationships or only consider single-hop social relationships.

Keywords: Deep Reinforcement Learning (DRL), Graph Neural Network (GNN), Interactive Recommendation (IR)

References(48)

[1]
D. Mican, D. A. Sitar-Tut, and O. I. Moisescu, Perceived usefulness: A silver bullet to assure user data availability for online recommendation systems, Decis. Support Syst., vol. 139, p. 113420, 2020.
[2]
B. Pathak, R. Garfinkel, R. D. Gopal, R. Venkatesan, and F. Yin, Empirical analysis of the impact of recommender systems on sales, J. Manag. Inf. Syst., vol. 27, no. 2, pp. 159–188, 2010.
[3]
S. Zhou, X. Dai, H. Chen, W. Zhang, K. Ren, R. Tang, X. He, and Y. Yu, Interactive recommender system via knowledge graph-enhanced reinforcement learning, in Proc. 43rd Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, New York, NY, USA, 2020, pp. 179–188.
[4]
N. Rubens, M. Elahi, M. Sugiyama, and D. Kaplan, Active learning in recommender systems, in Recommender Systems Handbook, F. Ricci, L. Rokach, and B. Shapira, eds. Boston, MA, USA: Springer, 2015, pp. 809–846.
DOI
[5]
X. Zhao, W. Zhang, and J. Wang, Interactive collaborative filtering, in Proc. 22nd ACM Int. Conf. on Information and Knowledge Management, San Francisco, CA, USA, 2013, pp. 1411–1420.
[6]
L. Li, W. Chu, J. Langford, and R. E. Schapire, A contextual-bandit approach to personalized news article recommendation, in Proc. 19th Int. Conf. on World Wide Web, Raleigh, NC, USA, 2010, pp. 661–670.
[7]
H. Wang, Q. Wu, and H. Wang, Learning hidden features for contextual bandits, in Proc. 25th ACM Int. on Conf. on Information and Knowledge Management, Indianapolis, IND, USA, 2016, pp. 1633–1642.
[8]
H. Wang, Q. Wu, and H. Wang, Factorization bandits for interactive recommendation, in Proc. 31st AAAI Conf. on Artificial Intelligence, San Francisco, CA, USA, 2017, pp. 2695–2702.
[9]
K. Zhu and T. Zhang, Deep reinforcement learning based mobile robot navigation: A review, Tsinghua Science and Technology, vol. 26, no. 5, pp. 674–691, 2021.
[10]
Y. Xiao, G. Niu, L. Xiao, Y. Ding, S. Liu, and Y. Fan, Reinforcement learning based energy-efficient internet-of-things video transmission, Intelligent and Converged Networks, vol. 1, no. 3, pp. 258–270, 2020.
[11]
W. Zhang, Z. Hou, X. Wang, Z. Xu, X. Liu, and F. Y. Wang, Parallel-data-based social evolution modeling, Tsinghua Science and Technology, vol. 26, no. 6, pp. 878–885, 2021.
[12]
G. Zheng, F. Zhang, Z. Zheng, Y. Xiang, N. J. Yuan, X. Xie, and Z. Li, DRN: A deep reinforcement learning framework for news recommendation, in Proc. 2018 World Wide Web Conf., Lyon, France, 2018, pp. 167–176.
[13]
X. Zhao, L. Xia, L. Zhang, Z. Ding, D. Yin, and J. Tang, Deep reinforcement learning for page-wise recommendations, in Proc. 12th ACM Conf. on Recommender Systems, Vancouver, Canada, 2018, pp. 95–103.
[14]
Y. Liu, Y. Xiao, Q. Wu, C. Miao, J. Zhang, B. Zhao, and H. Tang, Diversified interactive recommendation with implicit feedback, in Proc. 34th AAAI Conf. on Artificial Intelligence, Palo Alto, CA, USA, 2020, pp. 4932–4939.
[15]
J. Gu, J. Wang, L. Zhang, Z. Yu, X. Xin, and Y. Liu, Spotlight: Hot target discovery and localization with crowdsourced photos, Tsinghua Science and Technology, vol. 25, no. 1, pp. 68–80, 2019.
[16]
C. Peng, C. Zhang, X. Xue, J. Gao, H. Liang, and Z. Niu, Cross-modal complementary network with hierarchical fusion for multimodal sentiment classification, Tsinghua Science and Technology, vol. 27, no. 4, pp. 664–679, 2022.
[17]
K. Yang, J. Zhu, and X. Guo, POI neural-rec model via graph embedding representation, Tsinghua Science and Technology, vol. 26, no. 2, pp. 208–218, 2021.
[18]
S. Deng, L. Huang, G. Xu, X. Wu, and Z. Wu, On deep learning for trust-aware recommendations in social networks, IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 5, pp. 1164–1177, 2017.
[19]
J. Tang, X. Hu, and H. Liu, Social recommendation: A review, Soc. Netw. Anal. Min., vol. 3, no. 4, pp. 1113–1133, 2013.
[20]
G. Guo, J. Zhang, and N. Yorke-Smith, TrustSVD: Collaborative filtering with both the explicit and implicit influence of user trust and of item ratings, in Proc. 29th AAAI Conf. on Artificial Intelligence, Austin, TX, USA, 2015, pp. 123–129.
[21]
L. Wu, P. Sun, R. Hong, Y. Ge, and M. Wang, Collaborative neural social recommendation, IEEE Trans. Syst Man Cybernet Syst, vol. 51, no. 1, pp. 464–476, 2021.
[22]
L. Wu, P. Sun, Y. Fu, R. Hong, X. Wang, and M. Wang, A neural influence diffusion model for social recommendation, in Proc. 42nd Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, Paris, France, 2019, pp. 235–244.
[23]
L. Wu, J. Li, P. Sun, R. Hong, Y. Ge, and M. Wang, DiffNet++: A neural influence and interest diffusion network for social recommendation, IEEE Trans. Knowl. Data Eng., vol. 34, no. 10, pp. 4753–4766, 2020.
[24]
Y. Liu, C. Liang, X. He, J. Peng, Z. Zheng, and J. Tang, Modelling high-order social relations for item recommendation, IEEE Trans. Knowl. Data Eng., vol. 34, no. 9, pp. 4385–4397, 2022.
[25]
M. Jamali and M. Ester, A matrix factorization technique with trust propagation for recommendation in social networks, in Proc. Fourth ACM Conf. on Recommender Systems, Barcelona, Spain, 2010, pp. 135–142.
[26]
Y. Koren, R. Bell, and C. Volinsky, Matrix factorization techniques for recommender Systems, Computer, vol. 42, no. 8, pp. 30–37, 2009.
[27]
Y. Koren, Factorization meets the neighborhood: A multifaceted collaborative filtering model, in Proc. 14th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, Las Vegas, NV, USA, 2008, pp. 426–434.
[28]
X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. S. Chua, Neural collaborative filtering, in Proc. 26th Int. Conf. on World Wide Web, Perth, Australia, 2017, pp. 173–182.
[29]
X. He, Z. He, J. Song, Z. Liu, Y. G. Jiang, and T. S. Chua, NAIS: Neural attentive item similarity model for recommendation, IEEE Trans. Knowl. Data Eng., vol. 30, no. 12, pp. 2354–2366, 2018.
[30]
X. Zhao, L. Zhang, Z. Ding, L. Xia, J. Tang, and D. Yin, Recommendations with negative feedback via pairwise deep reinforcement learning, in Proc. 24th ACM SIGKDD Int. Conf. on Knowledge Discovery & Data Mining, London, UK, 2018, pp. 1040–1048.
[31]
L. Zou, L. Xia, Y. Gu, X. Zhao, W. Liu, J. X. Huang, and D. Yin, Neural interactive collaborative filtering, in Proc. 43rd Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, Xi’an, China, 2020, pp. 749–758
[32]
Q. Wu, H. Wang, Q. Gu, and H. Wang, Contextual bandits in a collaborative environment, in Proc. 39th Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, Pisa, Italy, 2016, pp. 529–538.
[33]
X. Wang, S. C. H. Hoi, C. Liu, and M. Ester, Interactive social recommendation, in Proc. 2017 ACM on Conf. on Information and Knowledge Management, Singapore, 2017, pp. 357–366.
[34]
Y. Lei, Z. Wang, W. Li, H. Pei, and Q. Dai, Social attentive deep Q-networks for recommender systems, IEEE Trans. Knowl. Data Eng., vol. 34, no. 5, pp. 2443–2457, 2022.
[35]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, Attention is all you need, in Proc. 31st Int. Conf. on Neural Information Processing Systems, Long Beach, CA, USA, 2017, pp. 6000–6010.
[36]
S. Natarajan, S. Vairavasundaram, S. Natarajan, and A. H. Gandomi, Resolving data sparsity and cold start problem in collaborative filtering recommender system using Linked Open Data, Exp. Syst. Appl., vol. 149, pp. 113–248, 2020.
[37]
J. Wei, J. H. He, K. Chen, Y. Zhou, and Z. Y. Tang, Collaborative filtering and deep learning based recommendation system for cold start items, Exp. Syst. Appl., vol. 69, pp. 29–39, 2017.
[38]
H. S. Sheu, Z. Chu, D. Qi, and S. Li, Knowledge-guided article embedding refinement for session-based news recommendation, IEEE Trans. Neural Netw. Learn. Syst., .
[39]
Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, A comprehensive survey on graph neural networks, IEEE Trans. Neural Netw. Learn. Syst., vol. 32, no. 1, pp. 4–24, 2021.
[40]
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, Playing Atari with deep reinforcement learning, arXiv preprint arXiv: 1312.5602, 2013.
[41]
Z. Wang, T. Schaul, M. Hessel, H. Van Hasselt, M. Lanctot, and N. De Freitas, Dueling network architectures for deep reinforcement learning, in Proc. 33rd Int. Conf. on Machine Learning, New York, NY, USA, 2016, pp. 1995–2003.
[42]
Y. Yue and T. Joachims, Interactively optimizing information retrieval systems as a dueling bandits problem, in Proc. 26th Ann. Int. Conf. on Machine Learning, Montreal, Canada, 2009, pp. 1201–1208.
[43]
K. Cho, B. Van Merriëboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, Learning phrase representations using RNN encoder-decoder for statistical machine translation, in Proc. 2014 Conf. on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014, pp. 1724–1734.
[44]
R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, Boston, MA, USA: MIT Press, 2018.
[45]
H. Van Hasselt, A. Guez, and D. Silver, Deep reinforcement learning with double Q-Learning, in Proc. 30th AAAI Conf. on Artificial Intelligence, Phoenix, AZ, USA, 2016, pp. 2094–2100.
[46]
Z. Guo and H. Wang, A deep graph neural network-based mechanism for social recommendations, IEEE Trans. Ind. Inf., vol. 17, no. 4, pp. 2776–2783, 2021.
[47]
Y. Hu, Q. Da, A. Zeng, Y. Yu, and Y. Xu, Reinforcement learning to rank in e-commerce search engine: Formalization, analysis, and application, in Proc. 24th ACM SIGKDD Int. Conf. on Knowledge Discovery & Data Mining, London, UK, 2018, pp. 368–377.
[48]
H. Chen, X. Dai, H. Cai, W. Zhang, X. Wang, R. Tang, Y. Zhang, and Y. Yu, Large-scale interactive recommendation with tree-structured policy gradient, in Proc. 33rd AAAI Conf. on Artificial Intelligence, Honolulu, HI, USA, 2019, pp. 3312–3320.
Publication history
Copyright
Rights and permissions

Publication history

Received: 05 July 2022
Revised: 31 August 2022
Accepted: 17 October 2022
Published: 06 January 2023
Issue date: August 2023

Copyright

© The author(s) 2023.

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return