International Journal of Crowd Science 2021, 5(3): 228-238 https://doi.org/10.1108/IJCS-03-2021-0005

Research paper |

Open Access | Issue | Published: 05 August 2021

An embedded bandit algorithm based on agent evolution for cold-start problem

Show Author's Information Hide Author's Information Rui Qiu^¹, Wen Ji^²(

)

Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Science, Beijing, China

Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

Keywords:

Word2Vec, Intelligence evolution, Cold-start, Multi-armed bandit

Cite this article:

Qiu R, Ji W. An embedded bandit algorithm based on agent evolution for cold-start problem. International Journal of Crowd Science, 2021, 5(3): 228-238. https://doi.org/10.1108/IJCS-03-2021-0005

Download citation

EndNote(RIS)

BibTeX

634

Views

Downloads

Citations

Crossref

N/A

WoS

Scopus

N/A

CSCD

Abstract Full text About this article

Abstract

Purpose

Many recommender systems are generally unable to provide accurate recommendations to users with limited interaction history, which is known as the cold-start problem. This issue can be resolved by trivial approaches that select random items or the most popular one to recommend to the new users. However, these methods perform poorly in many cases. This paper aims to explore the problem that how to make accurate recommendations for the new users in cold-start scenarios.

Design/methodology/approach

In this paper, the authors propose embedded-bandit method, inspired by Word2Vec technique and contextual bandit algorithm. The authors describe user contextual information with item embedding features constructed by Word2Vec. In addition, based on the intelligence measurement model in Crowd Science, the authors propose a new evaluation method to measure the utility of recommendations.

Findings

The authors introduce Word2Vec technique for constructing user contextual features, which improved the accuracy of recommendations compared to traditional multi-armed bandit problem. Apart from this, using this study's intelligence measurement model, the utility also outperforms.

Practical implications

Improving the accuracy of recommendations during the cold-start phase can greatly raise user stickiness and increase user favorability, which in turn contributes to the commercialization of the app.

Originality/value

The algorithm proposed in this paper reflects that user contextual features can be represented by clicked items embedding vector.

Full text

Abstract

Full text

Outline

About this article

An embedded bandit algorithm based on agent evolution for cold-start problem

Show Author's information Hide Author's Information Rui Qiu^¹, Wen Ji^²(

)

Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Science, Beijing, China

Beijing Key Laboratory of Mobile Computing and Pervasive Device, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

Abstract

Purpose

Design/methodology/approach

Findings

Practical implications

Improving the accuracy of recommendations during the cold-start phase can greatly raise user stickiness and increase user favorability, which in turn contributes to the commercialization of the app.

Originality/value

The algorithm proposed in this paper reflects that user contextual features can be represented by clicked items embedding vector.

Keywords: Word2Vec, Intelligence evolution, Cold-start, Multi-armed bandit

References(17)

Agarwal, D., Chen, B.-C. and Elango, P. (2009a), “Spatio-temporal models for estimating click-through rate”, in Proceedings of the 18th International Conference on World Wide Web, pp. 21-30.https://doi.org/10.1145/1526709.1526713

DOI

Agarwal, D., Chen, B.-C., Elango, P., Motgi, N., Park, S.-T., Ramakrishnan, R., Roy, S. and Zachariah, J. (2009b), “Online models for content optimization”, Advances in Neural Information Processing Systems, pp. 17-24.

Google Scholar

Auer, P., Cesa-Bianchi, N. and Fischer, P. (2002), “Finite-time analysis of the multiarmed bandit problem”, Machine Learning, Vol. 47 Nos 2/3, pp. 235-256.

Google Scholar

Bechara, A., Damasio, H., Tranel, D. and Damasio, A.R. (2005), “The Iowa gambling task and the somatic marker hypothesis: some questions and answers”,Trends in Cognitive Sciences, Vol. 9 No. 4, pp. 159-162.

DOI Google Scholar

Gentile, C., Li, S. and Zappella, G. (2014), “Online clustering of bandits”, in International Conference on Machine Learning. PMLR, pp. 757-765.

Gentile, C., Li, S., Kar, P., Karatzoglou, A., Zappella, G. and Etrue, E. (2017), “On context-dependent clustering of bandits”, in International Conference on Machine Learning, PMLR, pp. 1253-1262.

Harper, F.M. and Konstan, J.A. (2015), “The Movielens datasets: history and context”, Acm Transactions on Interactive Intelligent Systems (TIIS), Vol. 5 No. 4, pp. 1-19.

DOI Google Scholar

Lashkari, Y., Metral, M. and Maes, P. (1994), “Collaborative interface agents”, In AAAI, Vol. 94, pp. 444-449.

Google Scholar

Li, L., Chu, W., Langford, J. and Schapire, R.E. (2010) “A contextual-bandit approach to personalized news article recommendation”, in Proceedings of the 19th International Conference on World Wide Web, pp. 661-670.https://doi.org/10.1145/1772690.1772758

DOI

Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013), “Efficient estimation of word representations in vector space”, arXiv preprint arXiv:1301.3781.

Nguyen, T.T. and Lauw, H.W. (2014), “Dynamic clustering of contextual multi-armed bandits”, in Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management, pp. 1959-1962.https://doi.org/10.1145/2661829.2662063

DOI

Nguyen, H.T., Mary, J. and Preux, P. (2014), “Cold-start problems in recommendation systems via contextual-bandit algorithms”.

Schein, A.I., Popescul, A., Ungar, L.H. and Pennock, D.M. (2002) “Methods and metrics for cold-start recommendations”, in Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 253-260.https://doi.org/10.1145/564376.564421

DOI

Schulz, E., Konstantinidis, E. and Speekenbrink, M. (2018), “Putting bandits into context: how function learning supports decision making”, Journal of Experimental Psychology. Learning, Memory, and Cognition, Vol. 44 No. 6, p. 927.

DOI Google Scholar

Steyvers, M., Lee, M.D. and Wagenmakers, E.-J. (2009), “A Bayesian analysis of human decision-making on bandit problems”, Journal of Mathematical Psychology, Vol. 53 No. 3, pp. 168-179.

DOI Google Scholar

Yang, Z., Liang, B. and Ji, W. (2021), “An intelligent end-edge-cloud architecture for visual IOT assisted healthcare systems”, IEEE Internet of Things Journal.https://doi.org/10.1109/JIOT.2021.3052778

DOI

Yue, Y., Hong, S.A. and Guestrin, C. (2012), “Hierarchical exploration for accelerating contextual bandits”.

About this article

Publication history

Acknowledgements

Rights and permissions

Publication history

Received: 02 March 2021

Revised: 06 May 2021

Accepted: 24 May 2021

Published: 05 August 2021

Issue date: November 2021

Copyright

Acknowledgements

This work is supported by the National Key R&D Program of China (2017YFB1400100), the National Natural Science Foundation of China (61572466) and the Beijing Natural Science Foundation (4202072).

Rights and permissions

Rui Qiu and Wen Ji. Published in International Journal of Crowd Science. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode