Journal Home > Volume 8 , Issue 1
International Journal of Crowd Science 2024, 8(1): 10-27 https://doi.org/10.26599/IJCS.2023.9100010
Open Access | Issue | Published: 27 February 2024

Analysis of Motivational Theories in Crowdsourcing Using Long Tail Theory: A Systematic Literature Review

Show Author's Information Hasan Humayun1,2( ),  Mohammad Nauman Malik1,  Masitah Ghazali2
Department of Software Engineering, National University of Modern Languages, Islamabad 44000, Pakistan
Malaysia-Japan International Institute of Technology, Universiti Teknologi Malaysia, Kuala Lumpur 81310, Malaysia
Keywords:
crowdsourcing, systematic literature review, motivational theories, long tail theory, crowd engagement
Cite this article:
Humayun H, Malik MN, Ghazali M. Analysis of Motivational Theories in Crowdsourcing Using Long Tail Theory: A Systematic Literature Review. International Journal of Crowd Science, 2024, 8(1): 10-27. https://doi.org/10.26599/IJCS.2023.9100010
Download citation
EndNote(RIS)
BibTeX

184

Views

20

Downloads

Citations

0

Crossref

N/A

WoS

0

Scopus

N/A

CSCD

Abstract Full text About this article

Motivational theories have been extensively studied in a wide range of fields, such as medical sciences, business, management, physiology, sociology, and particularly in the natural sciences. These theories are regarded as crucial in motivating online workers to engage in crowdsourcing. Nevertheless, there is a dearth of research on an overarching review of these theories. We performed a systematic literature review of peer-reviewed published studies focusing on motivational theories to identify popular theories and risks associated with nascent theories presented over the last decade in crowdsourcing. Based on a review of 91 papers from the domain of the natural sciences, we identified 35 motivational theories. The long tail theory helped us to identify the contribution of major influencing theories in a crowdsourcing environment. The results justify the long tail theory based on the Pareto principle of 80/20, which underlines the 20% of the popular motivation theories, namely self-determination, expectancy-value, game, gamification, behavior change, and incentive theory, as a cause of 80%. Similarly, we discussed the risks associated with 10 theories presented over the long tail, which have a frequency equal to 2. Understanding the significant impact, approximately 80%, of widely recognized motivational theories and their role in risk identification is crucial. This understanding can assist researchers in optimizing their results by effectively integrating these theories.


menu
Abstract
Full text
Outline
About this article

Analysis of Motivational Theories in Crowdsourcing Using Long Tail Theory: A Systematic Literature Review

Show Author's information Hasan Humayun1,2( ), Mohammad Nauman Malik1, Masitah Ghazali2
Department of Software Engineering, National University of Modern Languages, Islamabad 44000, Pakistan
Malaysia-Japan International Institute of Technology, Universiti Teknologi Malaysia, Kuala Lumpur 81310, Malaysia

Abstract

Motivational theories have been extensively studied in a wide range of fields, such as medical sciences, business, management, physiology, sociology, and particularly in the natural sciences. These theories are regarded as crucial in motivating online workers to engage in crowdsourcing. Nevertheless, there is a dearth of research on an overarching review of these theories. We performed a systematic literature review of peer-reviewed published studies focusing on motivational theories to identify popular theories and risks associated with nascent theories presented over the last decade in crowdsourcing. Based on a review of 91 papers from the domain of the natural sciences, we identified 35 motivational theories. The long tail theory helped us to identify the contribution of major influencing theories in a crowdsourcing environment. The results justify the long tail theory based on the Pareto principle of 80/20, which underlines the 20% of the popular motivation theories, namely self-determination, expectancy-value, game, gamification, behavior change, and incentive theory, as a cause of 80%. Similarly, we discussed the risks associated with 10 theories presented over the long tail, which have a frequency equal to 2. Understanding the significant impact, approximately 80%, of widely recognized motivational theories and their role in risk identification is crucial. This understanding can assist researchers in optimizing their results by effectively integrating these theories.

Keywords: crowdsourcing, systematic literature review, motivational theories, long tail theory, crowd engagement

References(156)

[1]
M. Zhao and A. V. D. Hoek, A brief perspective on microtask crowdsourcing workflows for interface design, in Proc. 2015 IEEE/ACM 2nd Int. Workshop on CrowdSourcing in Software Engineering, Florence, Italy, 2015, pp. 45–46.
DOI
[2]

N. Stewart, J. Chandler, and G. Paolacci, Crowdsourcing samples in cognitive science, Trends Cogn. Sci., vol. 21, no. 10, pp. 736–748, 2017.
DOI Google Scholar
[3]

R. Dzombak, S. Mouakkad, and K. Mehta, Motivations of women participating in a technology-based social entrepreneurship program, Adv. Eng. Educ., vol. 5, no. 1, pp. 1–28, 2016.
Google Scholar
[4]

D. B. Shank, Using crowdsourcing websites for sociological research: The case of Amazon mechanical Turk, Am. Sociol., vol. 47, no. 1, pp. 47–55, 2016.
DOI Google Scholar
[5]

C. Matschke, J. Moskaliuk, F. Bokhorst, T. Schümmer, and U. Cress, Motivational factors of information exchange in social information spaces, Comput. Hum. Behav., vol. 36, pp. 549–558, 2014.
DOI Google Scholar
[6]

M. Zhou and H. Wang, The role of rationality in motivating participation in social movements: The case of anti-Japanese demonstrations in China, Ration. Soc., vol. 30, no. 1, pp. 155–186, 2018.
DOI Google Scholar
[7]

R. M. Ryan and E. L. Deci, Intrinsic and extrinsic motivations: Classic definitions and new directions, Contemp. Educ. Psychol., vol. 25, no. 1, pp. 54–67, 2000.
DOI Google Scholar
[8]

R. M. Ryan and E. L. Deci, Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being, Am. Psychol., vol. 55, no. 1, pp. 68–78, 2000.
DOI Google Scholar
[9]

A. Wigfield and J. S. Eccles, Expectancy–value theory of achievement motivation, Contemp. Educ. Psychol., vol. 25, no. 1, pp. 68–81, 2000.
DOI Google Scholar
[10]
J. V. Neumann and O. Morgenstern, Theory of Games and Economic Behavior (60th Anniversary Commemorative Edition). Princeton, NJ, USA: Princeton University Press, 2007.
DOI
[11]

K. Crowston and I. Fagnot, Stages of motivation for contributing user-generated content: A theory and empirical test, Int. J. Hum. Comput. Stud., vol. 109, pp. 89–101, 2018.
DOI Google Scholar
[12]

J. Howe, The rise of crowdsourcing, Wired, vol. 14, no. 6, pp. 176–183, 2006.
Google Scholar
[13]

A. M. Land-Zandstra, J. L. A. Devilee, F. Snik, F. Buurmeijer, and J. M. V. D. Broek, Citizen science on a smartphone: Participants’ motivations and learning, Public Underst. Sci., vol. 25, no. 1, pp. 45–60, 2016.
DOI Google Scholar
[14]

P. Graboviy, Methods of motivation improvement and effectiveness increase on the example of construction industry enterprises, Procedia Eng., vol. 165, pp. 1520–1528, 2016.
DOI Google Scholar
[15]
D. C. Brabham, The four urban governance problem types suitable for crowdsourcing citizen participation, in Citizen E-Participation in Urban Governance: Crowdsourcing and Collaborative Creativity, C. N. Silva, ed. Hershey, PA, USA: IGI Global, 2013, pp. 50–68.
DOI
[16]

H. Ning, S. Dhelim, M. A. Bouras, A. Khelloufi, and A. Ullah, Cyber-syndrome and its formation, classification, recovery and prevention, IEEE Access, vol. 6, pp. 35501–35511, 2018.
DOI Google Scholar
[17]

P. Buckley and E. Doyle, Gamification and student motivation, Interact. Learn. Environ., vol. 24, no. 6, pp. 1162–1175, 2016.
DOI Google Scholar
[18]

D. H. L. Goh, E. P. P. Pe-Than, and C. S. Lee, Perceptions of virtual reward systems in crowdsourcing games, Comput. Hum. Behav., vol. 70, pp. 365–374, 2017.
DOI Google Scholar
[19]

Y. Zhang and X. Zhang, Incentive mechanism with task bundling for mobile crowd sensing, ACM Trans. Sens. Net., vol. 19, no. 3, pp. 1–23, 2023.
DOI Google Scholar
[20]

O. Feyisetan and E. Simperl, Social incentives in paid collaborative crowdsourcing, ACM Trans. Intell. Syst. Technol., vol. 8, no. 6, pp. 1–31, 2017.
DOI Google Scholar
[21]

J. Goncalves, S. Hosio, J. Rogstadius, E. Karapanos, and V. Kostakos, Motivating participation and improving quality of contribution in ubiquitous crowdsourcing, Comput. Netw., vol. 90, pp. 34–48, 2015.
DOI Google Scholar
[22]

M. L. Maehr and H. A. Meyer, Understanding motivation and schooling: Where we’ve been, where we are, and where we need to go, Educ. Psychol. Rev., vol. 9, no. 4, pp. 371–409, 1997.
DOI Google Scholar
[23]

S. O. Olusanya, S. A. Awotungase, and A. O. Oyebo, Motivation, an engine for organizational performance, a case study of Lagos State University, external system, IOSR J. Bus. Manag., vol. 6, no. 2, pp. 30–41, 2012.
DOI Google Scholar
[24]
C. N. Cofer, Motivacion Y Emocion. Bilbao, Spain: Desclee Brouwer, 1976.
[25]
R. F. Thompson, Introduction to Physiological Psychology. New York, NY, USA: Harper and Row, 1975.
[26]
R. J. Sternberg and K. Sternberg, Cognitive Psychology. Boston, MA, USA: Wadsworth Publishing, 2016.
[27]
F. Widyahastuti, D. Fransiskus, and V. U. Tjhin, How active are K-12 students using Edmodo as online motivation, interaction and collaboration tools for learning process? in Proc. 2017 10th Int. Conf. Human System Interactions (HSI), Ulsan, Republic of Korea, 2017, pp. 94–97.
DOI
[28]

C. S. Chang, E. Z. F. Liu, H. Y. Sung, C. H. Lin, N. S. Chen, and S. S. Cheng, Effects of online college student’s internet self-efficacy on learning motivation and performance, Innov. Educ. Teach. Int., vol. 51, no. 4, pp. 366–377, 2014.
DOI Google Scholar
[29]
R. Kanfer, Motivation theory and industrial and organizational psychology, in Handbook of Industrial and Organizational Psychology, 1st ed. M. D. Dunnette and L. Hough, eds. Palo Alto, CA, USA: Consulting Psychologists Press, 1990, pp. 75–170.
[30]

L. Ferguson, S. Chan, M. Santelmann, and B. Tilt, Exploring participant motivations and expectations in a researcher-stakeholder engagement process: Willamette Water 2100, Landsc. Urban Plan., vol. 157, pp. 447–456, 2017.
DOI Google Scholar
[31]

E. D. Mekler, F. Brühlmann, A. N. Tuch, and K. Opwis, Towards understanding the effects of individual gamification elements on intrinsic motivation and performance, Comput. Hum. Behav., vol. 71, pp. 525–534, 2017.
DOI Google Scholar
[32]

S. Vanslambrouck, C. Zhu, K. Lombaerts, B. Philipsen, and J. Tondeur, Students’ motivation and subjective task value of participating in online and blended learning environments, Internet High. Educ., vol. 36, pp. 33–40, 2018.
DOI Google Scholar
[33]

K. S. Sankey and M. A. Machin, Employee participation in non-mandatory professional development—The role of core proactive motivation processes, Int. J. Train. Dev., vol. 18, no. 4, pp. 241–255, 2014.
DOI Google Scholar
[34]

C. S. Lee, D. H. L. Goh, H. Osop, S. C. J. Sin, and Y. L. Theng, Public services or private gains: Motives behind participation on a mobile crowdsourcing application in a smart city, Proc. Assoc. Info. Sci. Tech., vol. 54, no. 1, pp. 495–498, 2017.
DOI Google Scholar
[35]

S. Qiu, U. Gadiraju, M. V. Birk, and A. Bozzon, Using worker avatars to improve microtask crowdsourcing, Proc. ACM Hum.-Comput. Interact., vol. 5, p. 322, 2021.
DOI Google Scholar
[36]
I. Iacovides, C. Jennett, C. Cornish-Trestrail, and A. L. Cox, Do games attract or sustain engagement in citizen science? A study of volunteer motivations, in Proc. CHI '13 Extended Abstracts on Human Factors in Computing Systems, Paris, France, 2013, pp. 1101–1106.
DOI
[37]

N. Mairittha, T. Mairittha, P. Lago, and S. Inoue, CrowdAct: Achieving high-quality crowdsourced datasets in mobile activity recognition, Proc. ACM Interactive, Mobile, Wearable Ubiquitous Technol., vol. 5, no. 1, pp. 1–32, 2021.
DOI Google Scholar
[38]

A. Topîrceanu, Gamified learning: A role-playing approach to increase student in-class motivation, Procedia Comput. Sci., vol. 112, pp. 41–50, 2017.
DOI Google Scholar
[39]

M. Senkbeil and J. M. Ihme, Motivational factors predicting ICT literacy: First evidence on the structure of an ICT motivation inventory, Comput. Educ., vol. 108, pp. 145–158, 2017.
DOI Google Scholar
[40]

G. P. Latham and C. C. Pinder, Work motivation theory and research at the dawn of the twenty-first century, Annu. Rev. Psychol., vol. 56, no. 1, pp. 485–516, 2005.
DOI Google Scholar
[41]

Y. Sun, N. Wang, C. Yin, and J. X. Zhang, Understanding the relationships between motivators and effort in crowdsourcing marketplaces: A nonlinear analysis, Int. J. Inf. Manag., vol. 35, no. 3, pp. 267–276, 2015.
DOI Google Scholar
[42]
S. Karim, U. U. Shaikh, and Z. Asif, Motivating crowdworkers by managing expectations, in Proc. 2017 Int. Conf. Research and Innovation in Information Systems (ICRIIS), Langkawi, Malaysia, 2017, pp. 1–4.
DOI
[43]

W. H. Huang, W. Y. Huang, and J. Tschopp, Sustaining iterative game playing processes in DGBL: The relationship between motivational processing and outcome processing, Comput. Educ., vol. 55, no. 2, pp. 789–797, 2010.
DOI Google Scholar
[44]

R. Tinati, M. Luczak-Roesch, E. Simperl, and W. Hall, An investigation of player motivations in Eyewire, a gamified citizen science project, Comput. Hum. Behav., vol. 73, pp. 527–540, 2017.
DOI Google Scholar
[45]
B. Morschheuser, J. Hamari, and J. Koivisto, Gamification in crowdsourcing: A review, in Proc. 49th Hawaii Int. Conf. Syst. Sci., Koloa, HI, USA, 2016, pp. 4375–4384.
DOI
[46]
W. Y. Wu, Long tail theory in road safety reconstruction, in Proc. 14th COTA Int. Conf. Transportation Professionals, Changsha, China, 2014, pp. 2688–2696.
[47]
K. Katsuno, M. Matsubara, C. Watanabe, and A. Morishima, Improving reproducibility of crowdsourcing experiments, https://www.humancomputation.com/2019/assets/papers/119.pdf, 2019.
[48]
Y. Baba and H. Kashima, Statistical quality estimation for general crowdsourcing tasks, in Proc. 19th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, Chicago, IL, USA, 2013, pp. 554–562.
DOI
[49]

H. Humayun, M. Ghazali, and M. N. Malik, The role of motivational theories in the success of crowdsourcing engagement models: A review, J. Theor. Appl. Inf. Technol., vol. 101, no. 2, pp. 363–392, 2023.
Google Scholar
[50]

T. Aitamurto, H. Landemore, and J. S. Galli, Unmasking the crowd: Participants’ motivation factors, expectations, and profile in a crowdsourced law reform, Inf. Commun. Soc., vol. 20, no. 8, pp. 1239–1260, 2017.
DOI Google Scholar
[51]

K. Kim and S. J. Ahn, RETRACTED: The role of gamification in enhancing intrinsic motivation to use a loyalty program, J. Interact. Mark., vol. 40, pp. 41–51, 2017.
DOI Google Scholar
[52]
K. Spindeldreher and D. Schlagwein, What drives the crowd? A meta-analysis of the motivation of participants in crowdsourcing, presented at Pacific Asia Conf. Information Systems, Jiayi, China, 2016.
[53]

H. Ye and A. Kankanhalli, Investigating the antecedents of organizational task crowdsourcing, Inf. Manag., vol. 52, no. 1, pp. 98–110, 2015.
DOI Google Scholar
[54]
L. Gaur, N. Z. Jhanjhi, S. Bakshi, and P. Gupta, Analyzing consequences of artificial intelligence on jobs using topic modeling and keyword extraction, in Proc. 2022 2nd Int. Conf. Innovative Practices in Technology and Management (ICIPTM), Gautam Buddha Nagar, India, 2022, pp. 435–440.
DOI
[55]
M. N. Malik and A. Ullah, Computational intelligence through human machine interaction in cyber world, in Cyber-Enabled Intelligence, H. Ning, L. Chen, A. Ullah, and X. Luo, eds. New York, NY, USA: Taylor & Francis, 2019, pp. 141–148.
DOI
[56]

M. Hosseini, A. Shahri, K. Phalp, J. Taylor, and R. Ali, Crowdsourcing: A taxonomy and systematic mapping study, Comput. Sci. Rev., vol. 17, pp. 43–69, 2015.
DOI Google Scholar
[57]

E. Estellés-Arolas and F. González-Ladrón-de-Guevara, Towards an integrated crowdsourcing definition, J. Inf. Sci., vol. 38, no. 2, pp. 189–200, 2012.
DOI Google Scholar
[58]
T. Erickson, Some thoughts on a framework for crowdsourcing, presented at CHI 2011 Workshop on Crowdsourcing and Human Computation, New York, NY, USA, 2011.
[59]
J. A. Noble, Minority voices of crowdsourcing: Why we should pay attention to every member of the crowd, in Proc. CSCW’12 Conf. Computer Supported Cooperative Work, Seattle, WA, USA, 2012, pp. 179–182.
DOI
[60]

A. Afuah and C. L. Tucci, Crowdsourcing as a solution to distant search, Acad. Manag. Rev., vol. 37, no. 3, pp. 355–375, 2012.
DOI Google Scholar
[61]

M. Allahbakhsh, H. Amintoosi, and S. S. Kanhere, A game-theoretic approach to quality improvement in crowdsourcing tasks, Lecture Notes in Business Information Processing, vol. 234, pp. 116–130, 2018.
DOI Google Scholar
[62]

T. Aitamurto and H. Landemore, Crowdsourced deliberation: The case of the law on off-road traffic in Finland, Policy Internet, vol. 8, no. 2, pp. 174–196, 2016.
DOI Google Scholar
[63]
Y. Charalabidis, A. Triantafillou, V. Karkaletsis, and E. Loukis, Public policy formulation through non moderated crowdsourcing in social media, in Proc. Fourth IFIP WG 8.5 International Conference on Electronic Participation, Kristiansand, Norway, 2012, pp. 156–169.
DOI
[64]

S. Hosio, J. Goncalves, V. Kostakos, and J. Riekki, Crowdsourcing public opinion using urban pervasive technologies: Lessons from real-life experiments in Oulu, Policy Internet, vol. 7, no. 2, pp. 203–222, 2015.
DOI Google Scholar
[65]

R. S. Nadange, Customer perception about “crowdsourcing” within the suburbs of Mumbai, Procedia Econ. Finance, vol. 11, pp. 268–275, 2014.
DOI Google Scholar
[66]

M. Paulini, M. L. Maher, and P. Murty, Motivating participation in online innovation communities, Int. J. Web Based Communities, vol. 10, no. 1, pp. 94–114, 2014.
DOI Google Scholar
[67]

S. L. Alam and J. Campbell, Temporal motivations of volunteers to participate in cultural crowdsourcing work, Inf. Syst. Res., vol. 28, no. 4, pp. 744–759, 2017.
DOI Google Scholar
[68]
K. R. Lakhani and R. G. Wolf, Why hackers do what they do: Understanding motivation and effort in free/open source software projects, SSRN Electron. J., doi: 10.2139/ssrn.443040.
DOI
[69]

J. Webster and R. T. Watson, Analyzing the past to prepare for the future: Writing a literature review, MIS Q., vol. 26, no. 2, pp. xiii–xxiii, 2002.
Google Scholar
[70]

J. V. Brocke, A. Simons, K. Riemer, B. Niehaves, R. Plattfaut, and A. Cleven, Standing on the shoulders of giants: Challenges and recommendations of literature search in information systems research, Commun. Assoc. Inf. Syst., vol. 37, pp. 205–224, 2015.
DOI Google Scholar
[71]

B. Kitchenham and P. Brereton, A systematic review of systematic review process research in software engineering, Inf. Softw. Technol., vol. 55, no. 12, pp. 2049–2075, 2013.
DOI Google Scholar
[72]

D. Schlagwein and N. Bjorn-Andersen, Organizational learning with crowdsourcing: The revelatory case of LEGO, J. Assoc. Inf. Syst., vol. 15, no. 11, pp. 754–778, 2014.
DOI Google Scholar
[73]

C. Okoli, A guide to conducting a standalone systematic literature review, Commun. Assoc. Inf. Syst., vol. 37, no. 1, pp. 879–910, 2015.
DOI Google Scholar
[74]

J. F. Wolfswinkel, E. Furtmueller, and C. P. M. Wilderom, Using grounded theory as a method for rigorously reviewing literature, Eur. J. Inf. Syst., vol. 22, no. 1, pp. 45–55, 2013.
DOI Google Scholar
[75]
R. A. J. D. Vries, K. P. Truong, S. Kwint, C. H. C. Drossaert, and V. Evers, Crowd-designed motivation: Motivational messages for exercise adherence based on behavior change theory, in Proc. 34th Annual CHI Conf. Human Factors in Computing Systems, San Jose, CA, USA, 2016, pp. 297–308.
DOI
[76]
R. A. J. de Vries, K. P. Truong, and V. Evers, Crowd-designed motivation: Combining personality and the transtheoretical model, in International Conference on Persuasive Technology, A. Meschtscherjakov, B. De Ruyter, V. Fuchsberger, M. Murer, and M. Tscheligi, eds. Cham, Switzerland: Springer, 2016, pp. 41–52.
DOI
[77]

P. Dai, Mausam, and D. Weld, Decision-theoretic control of crowd-sourced workflows, Proc. AAAI Conf. Artif. Intell., vol. 24, no. 1, pp. 1168–1174, 2010.
DOI Google Scholar
[78]
Y. Kim, D. Gergle, and H. Zhang, Hit-or-wait: Coordinating opportunistic low-effort contributions to achieve global outcomes in on-the-go crowdsourcing, in Proc. 2018 CHI Conf. Human Factors in Computing Systems, Montreal, Canada, 2018, pp. 1–12.
DOI
[79]

Y. Sun, Y. Fang, and K. H. Lim, Understanding sustained participation in transactional virtual communities, Decis. Support. Syst., vol. 53, no. 1, pp. 12–22, 2012.
DOI Google Scholar
[80]

D. Li and L. Hu, Exploring the effects of reward and competition intensity on participation in crowdsourcing contests, Electron. Mark., vol. 27, no. 3, pp. 199–210, 2017.
DOI Google Scholar
[81]

Y. Moshfeghi, A. F. Huertas-Rosero, and J. M. Jose, A game-theory approach for effective crowdsource-based relevance assessment, ACM Trans. Intell. Syst. Technol., vol. 7, no. 4, pp. 1–25, 2016.
DOI Google Scholar
[82]
Y. He, L. Sun, Z. Li, H. Li, and X. Cheng, An optimal privacy-preserving mechanism for crowdsourced traffic monitoring, in Proc. 10th ACM Int. Workshop on Foundations of Mobile Computing, Philadelphia, PA, USA, 2014, pp. 11–18.
DOI
[83]
M. H. Cheung, R. Southwell, F. Hou, and J. Huang, Distributed time-sensitive task selection in mobile crowdsensing, in Proc. 16th ACM Int. Symp. on Mobile Ad Hoc Networking and Computing, Hangzhou, China, 2015, pp. 157–166.
DOI
[84]
S. N. S. Gaikwad, D. Morina, A. Ginzberg, C. Mullings, S. Goyal, D. Gamage, C. Diemert, M. Burton, S. Zhou, M. Whiting, et al., Boomerang: Rebounding the consequences of reputation feedback on crowdsourcing platforms, in Proc. 29th Annual Symp. User Interface Software and Technology, Tokyo, Japan, 2016, pp. 625–637.
DOI
[85]
H. A. Soufiani, D. M. Chickering, D. X. Charles, and D. C. Parkes, Approximating the shapley value via multi-issue decompositions, in Proc. Int. Conf. Autonomous Agents and Multi-Agent Systems, Paris, France, 2014, pp. 1209–1216.
[86]

D. E. O’Leary, An empirical analysis of information search and information sharing in crowdsourcing data analytic contests, Decis. Support Syst., vol. 120, pp. 1–13, 2019.
DOI Google Scholar
[87]

Y. Wang, Z. Cai, G. Yin, Y. Gao, X. Tong, and G. Wu, An incentive mechanism with privacy protection in mobile crowdsourcing systems, Comput. Netw., vol. 102, pp. 157–171, 2016.
DOI Google Scholar
[88]

J. H. Panchal, Z. Sha, and K. N. Kannan, Understanding design decisions under competition using games with information acquisition and a behavioral experiment, J. Mech. Des., vol. 139, no. 9, p. 091402, 2017.
DOI Google Scholar
[89]

W. Wu, W. T. Tsai, and W. Li, An evaluation framework for software crowdsourcing, Front. Comput. Sci., vol. 7, no. 5, pp. 694–709, 2013.
DOI Google Scholar
[90]

X. Xu, W. Wu, Y. Wang, and Y. Wu, Software crowdsourcing for developing software-as-a-service, Front. Comput. Sci., vol. 9, no. 4, pp. 554–565, 2015.
DOI Google Scholar
[91]
H. Zhang, Y. Wu, and W. Wu, Analyzing developer behavior and community structure in software crowdsourcing, in Information Science and Applications, K. J. Kim, ed. Berlin, Germany: Springer, 2015, pp. 981–988.
DOI
[92]

T. Chan, R. P. Gauthier, A. Suarez, N. F. Sia, and J. R. Wallace, Merlynne: Motivating peer-to-peer cognitive behavioral therapy with a serious game, Proc. ACM Hum.-Comput. Interact., vol. 5, p. 250, 2021.
DOI Google Scholar
[93]

M. Sailer, J. U. Hense, S. K. Mayr, and H. Mandl, How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction, Comput. Hum. Behav., vol. 69, pp. 371–380, 2017.
DOI Google Scholar
[94]

R. Yusri, A. Abusitta, and E. Aïmeur, Teens-online: A game theory-based collaborative platform for privacy education, Int. J. Artif. Intell. Educ., vol. 31, no. 4, pp. 726–768, 2021.
DOI Google Scholar
[95]

A. C. G. Santos, W. Oliveira, J. Hamari, L. Rodrigues, A. M. Toda, P. T. Palomino, and S. Isotani, The relationship between user types and gamification designs, User Modeling User Adapt. Interact., vol. 31, no. 5, pp. 907–940, 2021.
DOI Google Scholar
[96]

D. C. Brabham, Motivations for participation in a crowdsourcing application to improve public engagement in transit planning, J. Appl. Commun. Res., vol. 40, no. 3, pp. 307–328, 2012.
DOI Google Scholar
[97]

I. Fedorenko, P. Berthon, and T. Rabinovich, Crowded identity: Managing crowdsourcing initiatives to maximize value for participants through identity creation, Bus. Horizons, vol. 60, no. 2, pp. 155–165, 2017.
DOI Google Scholar
[98]

J. Ren, P. Ozturk, and W. Yeoh, Online crowdsourcing campaigns: Bottom-up versus top-down process model, J. Comput. Inf. Syst., vol. 59, no. 3, pp. 266–276, 2019.
DOI Google Scholar
[99]

J. Pang and Z. Liu, Motivation system of crowdsourcing community from a supply chain perspective, Math. Probl. Eng., vol. 2016, pp. 1–8, 2016.
DOI Google Scholar
[100]

P. B. Goes, C. Guo, and M. Lin, Do incentive hierarchies induce user effort? Evidence from an online knowledge exchange, Inf. Syst. Res., vol. 27, no. 3, pp. 497–516, 2016.
DOI Google Scholar
[101]

Y. Kong and G. Schoenebeck, An information theoretic framework for designing information elicitation mechanisms that reward truth-telling, ACM Trans. Econ. Comput., vol. 7, no. 1, pp. 1–33, 2016.
DOI Google Scholar
[102]

Y. Tran, M. Yonatany, and V. Mahnke, Crowdsourced translation for rapid internationalization in cyberspace: A learning perspective, Int. Bus. Rev., vol. 25, no. 2, pp. 484–494, 2016.
DOI Google Scholar
[103]

Y. Ruan, P. Zhang, L. Alfantoukh, and A. Durresi, Measurement theory-based trust management framework for online social communities, ACM Trans. Internet Technol., vol. 17, no. 2, pp. 1–24, 2017.
DOI Google Scholar
[104]
C. Preist, E. Massung, and D. Coyle, Competing or aiming to be average? Normification as a means of engaging digital volunteers, in Proc. 17th ACM Conf. Computer Supported Cooperative Work and Social Computing, Baltimore, MD, USA, 2014, pp. 1222–1233.
DOI
[105]

H. Y. S. Tsai, M. Jiang, S. Alhabash, R. LaRose, N. J. Rifon, and S. R. Cotten, Understanding online safety behaviors: A protection motivation theory perspective, Comput. Secur., vol. 59, pp. 138–150, 2016.
DOI Google Scholar
[106]

Q. Xu, Q. Huang, T. Jiang, B. Yan, W. Lin, and Y. Yao, HodgeRank on random graphs for subjective video quality assessment, IEEE Trans. Multimed., vol. 14, no. 3, pp. 844–857, 2012.
DOI Google Scholar
[107]
A. Barashev and G. Li, Personal trait predicting work engagement in crowdsourcing through achievement goals: Mediation analyses, in Proc. 8th Int. Conf. E-business, Management and Economics, Birmingham, UK, 2017, pp. 28–32.
DOI
[108]
A. Barashev and G. Li, Solvers’ motivation in crowdsourcing platform: Examining the impacts of reward sensitivity, and achievement goals factors, in Proc. 2018 9th Int. Conf. E-business, Management and Economics, Waterloo, Canada, 2018, pp. 80–85.
DOI
[109]

L. Posch, A. Bleier, C. M. Lechner, D. Danner, F. Flöck, and M. Strohmaier, Measuring motivations of crowdworkers, ACM Trans. Soc. Comput., vol. 2, no. 2, pp. 1–34, 2019.
DOI Google Scholar
[110]
B. Naderi, I. Wechsung, T. Polzehl, and S. Möller, Development and validation of extrinsic motivation scale for crowdsourcing micro-task platforms, in Proc. 2014 Int. ACM Workshop on Crowdsourcing for Multimedia, Orlando, FL, USA, 2014, pp. 31–36.
DOI
[111]

M. G. Martinez, Inspiring crowdsourcing communities to create novel solutions: Competition design and the mediating role of trust, Technol. Forecast. Soc. Change, vol. 117, pp. 296–304, 2017.
DOI Google Scholar
[112]

M. Heo and N. Toomey, Supporting sustained willingness to share knowledge with visual feedback, Comput. Hum. Behav., vol. 54, pp. 388–396, 2016.
DOI Google Scholar
[113]

A. Flostrand, T. Eriksson, and T. E. Brown, Better together—Harnessing motivations for energy utility crowdsourcing activities, Energy Res. Soc. Sci., vol. 48, pp. 57–65, 2019.
DOI Google Scholar
[114]
L. F. Zou, W. L. Ke, J. L. Zhang, and K. K. Wei, User creativity in crowdsourcing community: From extrinsic motivation perspective, presented at Pacific Asia Conference on Information Systems (PACIS 2014), Chengdu, China, 2014.
[115]

H. W. Lim, N. Li, D. Fang, and C. Wu, Impact of safety climate on types of safety motivation and performance: Multigroup invariance analysis, J. Manag. Eng., vol. 34, no. 3, p. 04018002, 2018.
DOI Google Scholar
[116]
M. J. Prince, K. E. K. Nottis, M. A. Vigeant, C. Kim, and E. Jablonski, The effect of course type on engineering undergraduates’ situational motivation and curiosity, in Proc. 2016 ASEE Annual Conf. & Exposition, New Orleans, LA, USA, 2016, p. 15714.
[117]

Y. C. Zhao and Q. Zhu, Effects of extrinsic and intrinsic motivation on participation in crowdsourcing contest: A perspective of self-determination theory, Online Inf. Rev., vol. 38, no. 7, pp. 896–917, 2014.
Google Scholar
[118]

B. Xu and D. Li, An empirical study of the motivations for content contribution and community participation in Wikipedia, Inf. Manag., vol. 52, no. 3, pp. 275–286, 2015.
DOI Google Scholar
[119]

J. B. Gassenheimer, J. A. Siguaw, and G. L. Hunter, Exploring motivations and the capacity for business crowdsourcing, AMS Rev., vol. 3, no. 4, pp. 205–216, 2013.
DOI Google Scholar
[120]
C. Schneider and F. V. Briel, Crowdsourcing large-scale ecological monitoring: Identifying design principles to motivate contributors, in Building Sustainable Information Systems, H. Linger, J. Fisher, A. Barnden, C. Barry, M. Lang, and C. Schneider, eds. New York, NY, USA: Springer, 2013, pp. 509–518.
DOI
[121]
P. Kaveti and M. N. Akbar, Role of intrinsic motivation in user interface design to enhance worker performance in Amazon MTurk, in Proc. 13th ACM Int. Conf. PErvasive Technologies Related to Assistive Environments, Corfu, Greece, 2020, pp. 34–40.
DOI
[122]

M. Wang, J. Wang, and W. N. Zhang, How to enhance solvers’ continuance intention in crowdsourcing contest: The role of interactivity and fairness perception, Online Information Review, vol. 44, no. 1, pp. 238–257, 2020.
DOI Google Scholar
[123]

M. M. Alhammah, L. Hajar, S. Alshathry, and M. Alqasabi, Motivational factors impacting the use of citizen reporting applications in Saudi Arabia: The case of balagh application, Int. J. Adv. Comput. Sci. Appl., vol. 12, no. 6, pp. 264–272, 2021.
DOI Google Scholar
[124]

L. Zou, S. Wei, W. Ke, and K. K. Wei, Creativity of participants in crowdsourcing communities: The effects of promotion focus and extrinsic motivation, J. Database Manag., vol. 31, no. 3, pp. 40–66, 2020.
DOI Google Scholar
[125]
U. Endriss and R. Fernández, Collective annotation of linguistic resources: Basic principles and a formal model, in Proc. 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria, 2013, pp. 539–549.
DOI
[126]
C. S. Lee, D. H. L. Goh, and H. Osop, To help without expectation: Investigating social exchanges on a mobile crowdsourcing platform for a smart city, in Proc. 2019 ACM/IEEE Joint Conf. Digital Libraries (JCDL), Champaign, IL, USA, 2019, pp. 416–417.
DOI
[127]

H. Ye and A. Kankanhalli, Solvers’ participation in crowdsourcing platforms: Examining the impacts of trust, and benefit and cost factors, J. Strateg. Inf. Syst., vol. 26, no. 2, pp. 101–117, 2017.
DOI Google Scholar
[128]
K. Starbird and L. Palen, “Voluntweeters”: Self-organizing by digital volunteers in times of crisis, in Proc. Human Factors in Computing Systems, Vancouver, Canada, 2011, pp. 1071–1080.
DOI
[129]

A. Aleta and Y. Moreno, The dynamics of collective social behavior in a crowd controlled game, EPJ Data Sci., vol. 8, no. 1, p. 22, 2019.
DOI Google Scholar
[130]

D. Renard and J. G. Davis, Social interdependence on crowdsourcing platforms, J. Bus. Res., vol. 103, pp. 186–194, 2019.
DOI Google Scholar
[131]

T. Trust, Motivation, empowerment, and innovation: Teachers’ beliefs about how participating in the edmodo math subject community shapes teaching and learning, J. Res. Technol. Educ., vol. 49, nos. 1&2, pp. 16–30, 2017.
DOI Google Scholar
[132]

M. Wilson, Where is the power in numbers? Understanding firm and consumer power when crowdsourcing, Bus. Horizons, vol. 61, no. 4, pp. 545–554, 2018.
DOI Google Scholar
[133]
A. R. Shahid and A. Elbanna, Who is in control in crowdsourcing initiatives? An examination of the case of crowdmapping, in Proc. 7th Scandinavian Conf. Information Systems, Ljungskile, Sweden, 2016, pp. 135–148.
DOI
[134]

N. Alomar, M. Alsaleh, and A. Alarifi, Uncovering the predictors of unsafe computing behaviors in online crowdsourcing contexts, Comput. Secur., vol. 85, pp. 300–312, 2019.
DOI Google Scholar
[135]
C. B. Jackson, C. Østerlund, G. Mugar, K. D. Hassman, and K. Crowston, Motivations for sustained participation in crowdsourcing: Case studies of citizen science on the role of talk, in Proc. 2015 48th Hawaii Int. Conf. System Sciences, Kauai, HI, USA, 2015, pp. 1624–1634.
DOI
[136]
A. Kittur, J. V. Nickerson, M. S. Bernstein, E. M. Gerber, A. Shaw, J. Zimmerman, M. Lease, and J. Horton, The future of crowd work, in Proc. 2013 Conf. Computer Supported Cooperative Work, Antonio, TX, USA, 2013, pp. 1301–1318.
DOI
[137]

E. Bucher, C. Fieseler, and C. Lutz, What’s mine is yours (for a nominal fee)—Exploring the spectrum of utilitarian to altruistic motives for Internet-mediated sharing, Comput. Hum. Behav., vol. 62, pp. 316–326, 2016.
DOI Google Scholar
[138]
R. Piderit, S. Flowerday, and S. McLean, Motivating citizens to contribute to the smart city: A public safety case study, in Proc. First Int. Internet of Things Summit, Rome, Italy, 2015, pp. 131–136.
DOI
[139]
L. Zhang and H. Zhang, Research of crowdsourcing model based on case study, in Proc. 8th Int. Conf. Service Systems and Service Management (ICSSSM’11), Tianjin, China, 2011, pp. 1–5.
[140]
A. Barashev and G. Li, Content factor analysis of crowd workers’ satisfaction, in Proc. 5th Int. Conf. E-Society, E-Learning and E-Technologies, Vienna, Austria, 2019, pp. 34–38.
DOI
[141]
J. Guo, J. Wang, and Z. Yan, Motivation and factors effecting the participation behavior in the urban crowdsourcing logistics: Evidence from China, in Proc. 10th Int. Conf. E-Education, E-Business, E-Management and E-Learning, Tokyo, Japan, 2019, pp. 334–341.
DOI
[142]
D. Guo, Q. Li, S. Liu, and H. Chai, Research on the ecology model of crowdfunding and crowdsourcing for digital education service and its applications in China, in Proc. 2019 4th Int. Conf. Distance Education and Learning, Shanghai, China, 2019, pp. 147–152.
DOI
[143]
S. Gilbert, Motivations for participating in online initiatives: Exploring contributory behaviour across initiative types, in Proc. CSCW’17 Companion: Companion of the 2017 ACM Conf. Computer Supported Cooperative Work and Social Computing, Portland, OR, USA, 2017, pp. 65–68.
DOI
[144]
J. Howe, Why the Power of the Crowd is Driving the Future of Business. New York, NY, USA: Crown Publishing Group, 2008.
[145]
R. A. J. D. Vries, K. P. Truong, and V. Evers, Crowd-designed motivation: Combining personality and the transtheoretical model, in Proc. 11th Int. Conf. Persuasive Technology, Salzburg, Austria, 2016, pp. 41–52.
DOI
[146]

F. Guay, R. J. Vallerand, and C. Blanchard, On the assessment of situational intrinsic and extrinsic motivation: The situational motivation scale (SIMS), Motiv. Emot., vol. 24, no. 3, pp. 175–213, 2000.
DOI Google Scholar
[147]
C. Anderson, The Long Tail, Revised and Updated Edition: Why the Future of Business Is Selling Less of More. London, UK: Random House Business, 2008.
[148]

Z. Jiang, Y. Huang, and D. R. Beil, The role of feedback in dynamic crowdsourcing contests: A structural empirical analysis, Manag. Sci., vol. 68, no. 7, pp. 4755–5555, 2022.
DOI Google Scholar
[149]

A. Ihl, K. S. Strunk, and M. Fiedler, The mediated effects of social support in professional online communities on crowdworker engagement in micro-task crowdworking, Comput. Hum. Behav., vol. 113, p. 106482, 2020.
DOI Google Scholar
[150]
Y. Zhao, K. Zheng, J. Guo, B. Yang, T. B. Pedersen, and C. S. Jensen, Fairness-aware task assignment in spatial crowdsourcing: Game-theoretic approaches, in Proc. 2021 IEEE 37th Int. Conf. Data Engineering (ICDE), Chania, Greece, 2021, pp. 265–276.
DOI
[151]
H. J. Khasraghi, X. Wang, and R. Hirschheim, Sustaining participant involvement in crowdsourcing contests through collaboration, presented at 23rd Americas Conf. Information Systems, Boston, MA, USA, 2017.
[152]

A. K. Abdulkareem, and R. M. Ramli, Does trust in e-government influence the performance of e-government? An integration of information system success model and public value theory, Transform Gov.: People, Process Policy, vol. 16, no. 1, pp. 1–17, 2022.
DOI Google Scholar
[153]

Y. Feng, H. J. Ye, Y. Yu, C. Yang, and T. Cui, Gamification artifacts and crowdsourcing participation: Examining the mediating role of intrinsic motivations, Comput. Hum. Behav., vol. 81, pp. 124–136, 2018.
DOI Google Scholar
[154]

T. Bakici, Comparison of crowdsourcing platforms from social-psychological and motivational perspectives, Int. J. Inf. Manag., vol. 54, p. 102121, 2020.
DOI Google Scholar
[155]

E. N. Moghaddam, A. Aliahmadi, M. Bagherzadeh, S. Markovic, M. Micevski, and F. Saghafi, Let me choose what I want: The influence of incentive choice flexibility on the quality of crowdsourcing solutions to innovation problems, Technovation, vol. 120, p. 102679, 2023.
DOI Google Scholar
[156]

M. Dai, Z. Su, Q. Xu, Y. Wang, and N. Lu, A trust-driven contract incentive scheme for mobile crowd-sensing networks, IEEE Trans. Veh. Technol., vol. 71, no. 2, pp. 1794–1806, 2022.
DOI Google Scholar
Publication history
Copyright
Rights and permissions

Publication history

Received: 27 February 2023
Revised: 22 May 2023
Accepted: 29 May 2023
Published: 27 February 2024
Issue date: March 2024

Copyright

© The author(s) 2024.

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return