Journal Home > Volume 3 , Issue 2
Purpose

Human or machine, which one is more intelligent and powerful for performing computing and processing tasks? Over the years, researchers and scientists have spent significant amounts of money and effort to answer this question. Nonetheless, despite some outstanding achievements, replacing humans in the intellectual tasks is not yet a reality. Instead, to compensate for the weakness of machines in some (mostly cognitive) tasks, the idea of putting human in the loop has been introduced and widely accepted. In this paper, the notion of collective hybrid intelligence as a new computing framework and comprehensive.

Design/methodology/approach

According to the extensive acceptance and efficiency of crowdsourcing, hybrid intelligence and distributed computing concepts, the authors have come up with the (complementary) idea of collective hybrid intelligence. In this regard, besides providing a brief review of the efforts made in the related contexts, conceptual foundations and building blocks of the proposed framework are delineated. Moreover, some discussion on architectural and realization issues are presented.

Findings

The paper describes the conceptual architecture, workflow and schematic representation of a new hybrid computing concept. Moreover, by introducing three sample scenarios, its benefits, requirements, practical roadmap and architectural notes are explained.

Originality/value

The major contribution of this work is introducing the conceptual foundations to combine and integrate collective intelligence of humans and machines to achieve higher efficiency and (computing) performance. To the best of the authors’ knowledge, this the first study in which such a blessing integration is considered. Therefore, it is believed that the proposed computing concept could inspire researchers toward realizing such unprecedented possibilities in practical and theoretical contexts.


menu
Abstract
Full text
Outline
About this article

Collective hybrid intelligence: towards a conceptual framework

Show Author's information Morteza Moradi1Mohammad Moradi2Farhad Bayat1( )Adel Nadjaran Toosi3
Department of Electrical Engineering, University of Zanjan, Zanjan, Iran
Young Researchers and Elite Club, Qazvin, Islamic Republic of Iran
Faculty of Information Technology, Monash University, Melbourne, Australia

Abstract

Purpose

Human or machine, which one is more intelligent and powerful for performing computing and processing tasks? Over the years, researchers and scientists have spent significant amounts of money and effort to answer this question. Nonetheless, despite some outstanding achievements, replacing humans in the intellectual tasks is not yet a reality. Instead, to compensate for the weakness of machines in some (mostly cognitive) tasks, the idea of putting human in the loop has been introduced and widely accepted. In this paper, the notion of collective hybrid intelligence as a new computing framework and comprehensive.

Design/methodology/approach

According to the extensive acceptance and efficiency of crowdsourcing, hybrid intelligence and distributed computing concepts, the authors have come up with the (complementary) idea of collective hybrid intelligence. In this regard, besides providing a brief review of the efforts made in the related contexts, conceptual foundations and building blocks of the proposed framework are delineated. Moreover, some discussion on architectural and realization issues are presented.

Findings

The paper describes the conceptual architecture, workflow and schematic representation of a new hybrid computing concept. Moreover, by introducing three sample scenarios, its benefits, requirements, practical roadmap and architectural notes are explained.

Originality/value

The major contribution of this work is introducing the conceptual foundations to combine and integrate collective intelligence of humans and machines to achieve higher efficiency and (computing) performance. To the best of the authors’ knowledge, this the first study in which such a blessing integration is considered. Therefore, it is believed that the proposed computing concept could inspire researchers toward realizing such unprecedented possibilities in practical and theoretical contexts.

Keywords:

Crowdsourcing, Human computation, Autonomous control, Collective machine intelligence, Human–machine collaboration, Hybrid intelligence
Received: 26 March 2019 Revised: 03 June 2019 Accepted: 11 July 2019 Published: 16 August 2019 Issue date: September 2019
References(111)

Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M. and Werthimer, D. (2002), “SETI@ home: an experiment in public-resource computing”, Communications of the Acm, Vol. 45 No. 11, pp. 56-61.

Atzori, L., Iera, A., Morabito, G. and Nitti, M. (2012), “The social internet of things (siot)–when social networks meet the internet of things: concept, architecture and network characterization”, Computer Networks, Vol. 56 No. 16, pp. 3594-3608.

Barbier, G., Zafarani, R., Gao, H., Fung, G. and Liu, H. (2012), “Maximizing benefits from crowdsourced data”, Computational and Mathematical Organization Theory, Vol. 18 No. 3, pp. 257-279.

Bargiela, A. and Pedrycz, W. (2016), “Granular computing”, in Handbook on Computational Intelligence, Vol. 1, Fuzzy Logic, Systems, Artificial Neural Networks, and Learning Systems, pp. 43-66.https://doi.org/10.1142/9789814675017_0002
DOI

Baruch, A., May, A. and Yu, D. (2016), “The motivations, enablers and barriers for voluntary participation in an online crowdsourcing platform”, Computers in Human Behavior, Vol. 64, pp. 923-931.

Bayat, F., Najafinia, S. and Aliyari, M. (2018), “Mobile robots path planning: electrostatic potential field approach”, Expert Systems with Applications, Vol. 100, pp. 68-78.

Beberg, A.L., Ensign, D.L., Jayachandran, G., Khaliq, S. and Pande, V.S. (2009), “Folding@ home: lessons from eight years of volunteer distributed computing”,Proceedings of International Parallel and Distributed Processing Symposium, Rome, Italy, pp. 1-8.https://doi.org/10.1109/IPDPS.2009.5160922
DOI

Bien, Z., Bang, W.C., Kim, D.Y. and Han, J.S. (2002), “Machine intelligence quotient: its measurements and applications”, Fuzzy Sets and Systems, Vol. 127 No. 1, pp. 3-16.

Bonabeau, E. (2009), “Decisions 2.0: the power of collective intelligence”, MIT Sloan Management Review, Article 45, Vol. 50 No. No. 2.

Bonomi, F., Milito, R., Zhu, J. and Addepalli, S. (2012), “Fog computing and its role in the internet of things”, Proceedings of the first edition of the MCC workshop on Mobile Cloud Computing, ACM; pp. 13-16.https://doi.org/10.1145/2342509.2342513
DOI
Boutsis, I. and Kalogeraki, V. (2016), ““Location privacy for crowdsourcing applications”, Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, ACM, pp. 694-705.https://doi.org/10.1145/2971648.2971741
DOI

Brabham, D.C. (2008), “Crowdsourcing as a model for problem solving: an introduction and cases”, Convergence: The International Journal of Research into New Media Technologies, Vol. 14 No. 1, pp. 75-90.

Breazeal, C., DePalma, N., Orkin, J., Chernova, S. and Jung, M. (2013), “Crowdsourcing human-robot interaction: new methods and system evaluation in a public environment”, Journal of Human-Robot Interaction, Vol. 2 No. 1, pp. 82-111.

Buchanan, L. and O'Connell, A. (2006), “A brief history of decision making”, Harvard Business Review, Vol. 84 No. 1, pp. 32-40.

Bundy, A. (2017), “Smart machines are not a threat to humanity”, Communications of the Acm, Vol. 60 No. 2, pp. 40-42.

Burnstein, E. and Berbaum, M.L. (1983), “Stages in group decision making: the decomposition of historical narratives”, Political Psychology, Vol. 4 No. 3, pp. 531-561.

Cassimatis, N.L. (2006), “A cognitive substrate for achieving human-level intelligence”, AI Magazine, Vol. 27 No. 2, pp. 45-45.

Chang, J.C., Amershi, S. and Kamar, E. (2017), “Revolt: collaborative crowdsourcing for labeling machine learning datasets”, Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM, pp. 2334-2346.https://doi.org/10.1145/3025453.3026044
DOI
Chen, S.C.Y. and Shen, M.C. (2019), “The fourth industrial revolution and the development of artificial intelligence”, Contemporary Issues in International Political Economy, Palgrave Macmillan, Singapore, pp. 333-346.https://doi.org/10.1007/978-981-13-6462-4_14
DOI

Chien, A., Calder, B., Elbert, S. and Bhatia, K. (2003), “Entropia: architecture and performance of an enterprise desktop grid system”,Journal of Parallel and Distributed Computing, Vol. 63 No. 5, pp. 597-610.

Chiu, C.M., Liang, T.P. and Turban, E. (2014), “What can crowdsourcing do for decision support?”, Decision Support Systems, Vol. 65, pp. 40-49.

Copeland, B.J. (2000), The Modern History of Computing, Zalta, E.N. (Ed.), Winter 2017 Edition The Stanford Encyclopedia of Philosophy, available at: https://plato.stanford.edu/archives/win2017/entries/computing-history/
Dai, P. and Weld, D.S. (2011), “Artificial intelligence for artificial intelligence”, Proceedings of Twenty-Fifth AAAI Conference on Artificial Intelligence.

Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B. and Allahbakhsh, M. (2018), “Quality control in crowdsourcing: a survey of quality attributes, assessment techniques, and assurance actions”, ACM Computing Surveys, Article 7, Vol. 51 No. 1.

Decker, M. (2000), “Replacing human beings by robots. How to tackle that perspective by technology assessment”, in Vision Assessment: Shaping Technology in 21st Century Society, Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 149-166.https://doi.org/10.1007/978-3-642-59702-2_7
DOI
Del Prado, G.M. (2015), “18 Artificial intelligence researchers reveal the profound changes coming to our lives”, Business Insider, available at: www.businessinsider.com/researchers-predictions-future-artificial-intelligence-2015-10 (accessed 10 September 2018).

Deng, W., Chen, R., Gao, J., Song, Y. and Xu, J. (2012), “A novel parallel hybrid intelligence optimization algorithm for a function approximation problem”, Computers and Mathematics with Applications, Vol. 63 No. 1, pp. 325-336.

Difallah, D.E., Demartini, G. and Cudré-Mauroux, P. (2012), “Mechanical cheat: spamming schemes and adversarial techniques on crowdsourcing platforms”, Proceedings of the First International Workshop on Crowdsourcing Web Search, pp. 26-30.https://doi.org/10.1145/2187836.2187900
DOI
Dobrev, D. (2012), A Definition of Artificial Intelligence”, arXiv preprint arXiv:1210.1568.
Dodge, S. and Karam, L. (2017), “A study and comparison of human and deep learning recognition performance under visual distortions”, Proceedings of the 26th International Conference on Computer Communication and Networks (ICCCN), IEEE, pp. 1-7.https://doi.org/10.1109/ICCCN.2017.8038465
DOI
Everitt, T. and Hutter, M. (2018), “Universal artificial intelligence”, In Foundations of Trusted Autonomy, Springer, Cham; pp. 15-46.https://doi.org/10.1007/978-3-319-64816-3_2
DOI

Fabisiak, T. and Danilecki, A. (2017), “Browser-based harnessing of voluntary computational power”, Foundations of Computing and Decision Sciences, Vol. 42 No. 1, pp. 3-42.

Feigenbaum, E.A. (2003), “Some challenges and grand challenges for computational intelligence”, Journal of the ACM ( Acm), Vol. 50 No. 1, pp. 32-40.

Feng, W., Yan, Z., Zhang, H., Zeng, K., Xiao, Y. and Hou, Y.T. (2017), “A survey on security, privacy, and trust in mobile crowdsourcing”, IEEE Internet of Things Journal, Vol. 5 No. 4, pp. 2971-2992.

Fleuret, F., Li, T., Dubout, C., Wampler, E.K., Yantis, S. and Geman, D. (2011), “Comparing machines and humans on a visual categorization test”, Proceedings of the National Academy of Sciences, Vol. 108 No. 43, pp. 17621-17625.

Folds, D.J. (2016), “Human executive control of autonomous systems: a conceptual framework”, Proceedings of IEEE International Symposium on Systems Engineering (ISSE), pp. 1-5.https://doi.org/10.1109/SysEng.2016.7753126
DOI
Gifford, C.M. (2009), “Collective machine learning: team learning and classification in multi-agent systems”, Doctoral dissertation, University of Kansas.
Guazzini, A., Vilone, D., Donati, C., Nardi, A. and Levnajić, Z. (2015), “Modeling crowdsourcing as collective problem solving”, Scientific reports, Vol. 5, Article number: 16557.https://doi.org/10.1038/srep16557
DOI

Gurkaynak, G., Yilmaz, I. and Haksever, G. (2016), “Stifling artificial intelligence: human perils”, Computer Law and Security Review, Vol. 32 No. 5, pp. 749-758.

Halmes, M. (2013), “Measurements of collective machine intelligence”, arXiv preprint arXiv:1306.6649.
Hernández-Orallo, J. and Minaya-Collado, N. (1998), “A formal definition of intelligence based on an intensional variant of algorithmic complexity”, Proceedings of International Symposium of Engineering of Intelligent Systems (EIS98), pp. 146-163.

Hibbard, B. (2001), “Super-intelligent machines”, ACM SIGGRAPH Computer Graphics, Vol. 35 No. 1, pp. 11-13.

Howe, J. (2006), “The rise of crowdsourcing”, Wired Magazine, Vol. 14 No. 6, pp. 1-4.

Huang, Y., Vir Singh, P. and Srinivasan, K. (2014), “Crowdsourcing new product ideas under consumer learning”, Management Science, Vol. 60 No. 9, pp. 2138-2159.

Huang, F.Y., Wang, K., An, Y. and Lasecki, W.S. (2017), “Towards hybrid intelligence for robotics”, Proceedings of The 5th Edition of the Collective Intelligence Conference.

Ilyashenko, A.S., Lukashin, A.A., Zaborovsky, V.S. and Lukashin, A.A. (2017), “Algorithms for planning resource-intensive computing tasks in a hybrid supercomputer environment for simulating the characteristics of a quantum rotation sensor and performing engineering calculations”, Automatic Control and Computer Sciences, Vol. 51 No. 6, pp. 426-434.

Jonathan, A., Ryden, M., Oh, K., Chandra, A. and Weissman, J. (2017), “Nebula: distributed edge cloud for data intensive computing”, IEEE Transactions on Parallel and Distributed Systems, Vol. 28 No. 11, pp. 3229-3242.

Kajino, H., Arai, H. and Kashima, H. (2014), “Preserving worker privacy in crowdsourcing”, Data Mining and Knowledge Discovery, Vol. 28 Nos 5/6, pp. 1314-1335.

Kamar, E. (2016), “Directions in hybrid intelligence: complementing AI systems with human intelligence”, Proceedings of IJCAI, pp. 4070-4073.
Karimi, H.A. (Ed.) (2004), “Telegeoinformatics: Location-Based Computing and Services”, CRC Press.https://doi.org/10.1201/b12395
DOI
Kennedy, J. (2006), “Swarm intelligence”, in Handbook of Nature-Inspired and Innovative Computing, Springer, pp. 187-219.https://doi.org/10.1007/0-387-27705-6_6
DOI

Kephart, J.O. and Chess, D.M. (2003), “The vision of autonomic computing”, Computer, Vol. 1, pp. 41-50.

Khoi, N., Casteleyn, S., Moradi, M. and Pebesma, E. (2018), “Do monetary incentives influence users’ behavior in participatory sensing?”, Sensors, Vol. 18 No. 5, Article 1426.

Klumpp, M., Hesenius, M., Meyer, O., Ruiner, C. and Gruhn, V. (2019), “Production logistics and human-computer interaction—state-of-the-art, challenges and requirements for the future”, The International Journal of Advanced Manufacturing Technology, doi: 10.1007/s00170-019-03785-0.

Kondo, D., Andrzejak, A. and Anderson, D.P. (2008), “On correlated availability in internet-distributed systems”, Proceedings of the 2008 9th IEEE/ACM International Conference on Grid Computing, IEEE, pp. 276-283.https://doi.org/10.1109/GRID.2008.4662809
DOI
Kube, C.R. and Zhang, H. (1992), “Collective robotic intelligence”, Proceedings of the Second International Conference on Simulation of Adaptive Behavior, pp. 460-468.
Larson, S.M. Snow, C.D. Shirts, M. and Pande, V.S. (2009), “Folding@ Home and Genome@ Home: Using distributed computing to tackle previously intractable problems in computational biology”, arXiv preprint arXiv:0901.0866.

Legg, S. and Hutter, M. (2007), “Universal intelligence: a definition of machine intelligence”, Minds and Machines, Vol. 17 No. 4, pp. 391-444.

Leimeister, J.M. (2010), “Collective intelligence”, Business and Information Systems Engineering, Vol. 2 No. 4, pp. 245-248.

Li, J., Pan, Z., Xu, J., Liang, B., Chen, Y. and Ji, W. (2018), “Quality-time-complexity universal intelligence measurement”, International Journal of Crowd Science, Vol. 2 No. 2, pp. 99-107.

Litman, L., Robinson, J. and Abberbock, T. (2017), “TurkPrime. com: a versatile crowdsourcing data acquisition platform for the behavioral sciences”, Behavior Research Methods, Vol. 49 No. 2, pp. 433-442.

Liu, J., Pacitti, E., Valduriez, P. and Mattoso, M. (2015), “A survey of data-intensive scientific workflow management”, Journal of Grid Computing, Vol. 13 No. 4, pp. 457-493.

Lu, H., Li, Y., Chen, M., Kim, H. and Serikawa, S. (2018), “Brain intelligence: go beyond artificial intelligence”, Mobile Networks and Applications, Vol. 23 No. 2, pp. 368-375.

Luo, S., Xia, H., Yoshida, T. and Wang, Z. (2009), “Toward collective intelligence of online communities: a primitive conceptual model”, Journal of Systems Science and Systems Engineering, Vol. 18 No. 2, pp. 203-221.

Mahoney, M.S. (1988), “The history of computing in the history of technology”, Ieee Annals of the History of Computing, Vol. 10 No. 2, pp. 113-125.

Makridakis, S. (2017), “The forthcoming artificial intelligence (AI) revolution: its impact on society and firms”, Futures, Vol. 90, pp. 46-60.

Maleszka, M. and Nguyen, N.T. (2015), “Integration computing and collective intelligence”, Expert Systems with Applications, Vol. 42 No. 1, pp. 332-340.

Mao, A., Kamar, E., Chen, Y., Horvitz, E., Schwamb, M.E., Lintott, C.J. and Smith, A.M. (2013), “Volunteering versus work for pay: incentives and tradeoffs in crowdsourcing”, Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing.

Maoz, Z. (1990), “Framing the national interest: the manipulation of foreign policy decisions in group settings”, World Politics, Vol. 43 No. 1, pp. 77-110.

Miailhe, N. and Hodes, C. (2017), “The third age of artificial intelligence. Field actions science reports”, The Journal of Field Actions, (Special Issue), Vol. 17, pp. 6-11.

Müller, V.C. and Bostrom, N. (2016), “Future progress in artificial intelligence: a survey of expert opinion”, In Fundamental Issues of Artificial Intelligence, Springer, Cham, pp. 555-572.https://doi.org/10.1007/978-3-319-26485-1_33
DOI

Nilsson, N.J. (2005), “Human-level artificial intelligence? Be serious!”, AI Magazine, Vol. 26 No. 4, pp. 68-68.

Nowak, S. and Rüger, S. (2010), “How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation”, Proceedings of the international conference on Multimedia Information Retrieval, ACM, pp. 557-566.https://doi.org/10.1145/1743384.1743478
DOI
Nushi, B., Kamar, E. and Horvitz, E. (2018), “Towards accountable AI: hybrid human-machine analyses for characterizing system failure”, Proceedings of the Sixth AAAI Conference on Human Computation and Crowdsourcing.

Ofli, F., Meier, P., Imran, M., Castillo, C., Tuia, D., Rey, N., Briant, J., Millet, P., Reinhard, F., Parkan, M. and Joost, S. (2016), “Combining human computing and machine learning to make sense of big (aerial) data for disaster response”, Big Data, Vol. 4 No. 1, pp. 47-59.

Pan, Y. (2016), “Heading toward artificial intelligence 2.0”, Engineering, Vol. 2 No. 4, pp. 409-413.

Pedreira, M.M. and Grigoras, C. (2017), “Scalable Global Grid catalogue for LHC Run3 and beyond”, arXiv preprint arXiv:1704.05272.

Peer, E., Brandimarte, L., Samat, S. and Acquisti, A. (2017), “Beyond the Turk: alternative platforms for crowdsourcing behavioral research”, Journal of Experimental Social Psychology, Vol. 70, pp. 153-163.

Pilz, D. and Gewald, H. (2013), “Does money matter? Motivational factors for participation in paid-and non-profit-crowdsourcing communities”, Wirtschaftsinformatik, Vol. 37, pp. 73-82.

Quinn, A.J. and Bederson, B.B. (2011), “Human computation: a survey and taxonomy of a growing field”, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp. 1403-1412.https://doi.org/10.1145/1978942.1979148
DOI

Rawat, A.S., Papailiopoulos, D.S., Dimakis, A.G. and Vishwanath, S. (2016), “Locality and availability in distributed storage”, IEEE Transactions on Information Theory, Vol. 62 No. 8, pp. 4481-4493.

Retelny, D., Robaszkiewicz, S., To, A., Lasecki, W.S., Patel, J., Rahmati, N., Doshi, T., Valentine, M. and Bernstein, M.S. (2014), “Expert crowdsourcing with flash teams”, Proceedings of the 27th annual ACM symposium on User Interface Software and Technology, ACM, pp. 75-85.https://doi.org/10.1145/2642918.2647409
DOI

Russell, S. (2017), “Artificial intelligence: the future is superintelligent”, Nature, Vol. 548 No. 7669, pp. 520-522.

Sadashiv, N. and Kumar, S.D. (2011), “Cluster, grid and cloud computing: a detailed comparison”,Proceedings of the 6th International Conference on Computer Science and Education (ICCSE), IEEE, pp. 477-482.https://doi.org/10.1109/ICCSE.2011.6028683
DOI

Sarathy, V. (2018), “Real world problem-solving”, Frontiers in Human Neuroscience, Vol. 12PMC6028615.

Schemmann, B., Herrmann, A.M., Chappin, M.M. and Heimeriks, G.J. (2016), “Crowdsourcing ideas: involving ordinary users in the ideation phase of new product development”, Research Policy, Vol. 45 No. 6, pp. 1145-1154.

Shi, W., Cao, J., Zhang, Q., Li, Y. and Xu, L. (2016), “Edge computing: vision and challenges”, IEEE Internet of Things Journal, Vol. 3 No. 5, pp. 637-646.

Singh, S.P., Nayyar, A., Kaur, H. and Singla, A. (2019), “Dynamic task scheduling using balanced VM allocation policy for fog computing platforms”, Scalable Computing: Practice and Experience, Vol. 20 No. 2, pp. 433-456.

Stabinger, S., Rodríguez-Sánchez, A. and Piater, J. (2016), “25 Years of CNNs: Can we compare to human abstraction capabilities?”, Proceedings of International Conference on Artificial Neural Networks, Springer, Cham, pp. 380-387.https://doi.org/10.1007/978-3-319-44781-0_45
DOI
Stahl, F., Gaber, M.M., Bramer, M. and Philip, S.Y. (2010), “Pocket data mining: towards collaborative data mining in mobile computing environments”, Proceedings of 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI), IEEE, Vol. 2, pp. 323-330.https://doi.org/10.1109/ICTAI.2010.118
DOI

Steinhardt, J., Valiant, G. and Charikar, M. (2016), “Avoiding imposters and delinquents: adversarial crowdsourcing and peer prediction”, In Advances in Neural Information Processing Systems, pp. 4439-4447.

Vaughan, J.W. (2017), “Making better use of the crowd: how crowdsourcing can advance machine learning research”, Journal of Machine Learning Research, Vol. 18, pp. 1-46.

Verhulst, S.G. (2018), “Where and when AI and CI meet: exploring the intersection of artificial and collective intelligence towards the goal of innovating how we govern”,AI and Society, Vol. 33 No. 2, pp. 293-297.

Vij, D. and Aggarwal, N. (2018), “Smartphone based traffic state detection using acoustic analysis and crowdsourcing”, Applied Acoustics, Vol. 138, pp. 80-91.

Vochin, M., Zoican, S. and Borcoci, E. (2018), “Intelligent vehicle navigation system with assistance and alerting capabilities”, Concurrency and Computation: Practice and Experience, p. e4402, available at: https://doi.org/10.1002/cpe.4402

Von Ahn, L. (2008), “Human computation”, Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, pp. 1-2.https://doi.org/10.1109/ICDE.2008.4497403
DOI

Wang, F.Y., Carley, K.M., Zeng, D. and Mao, W. (2007), “Social computing: from social informatics to social intelligence”, IEEE Intelligent Systems, Vol. 22 No. 2.

Wang, W., Johnston, B. and Williams, M.A. (2012), “Social networking for robots to share knowledge, skills and know-how”, Proceedings of International Conference on Social Robotics, Springer, Berlin, Heidelberg, pp. 418-427.https://doi.org/10.1007/978-3-642-34103-8_42
DOI

Weyer, J., Fink, R.D. and Adelt, F. (2015), “Human–machine cooperation in smart cars: an empirical investigation of the loss-of-control thesis”, Safety Science, Vol. 72, pp. 199-208.

Wiedermann, J. (2012), “Is there something beyond AI? Frequently emerging, but seldom answered questions about artificial Super-Intelligence”, Proceedings of the International Conference Beyond AI, pp. 76-86.
Wightman, D. (2010), “Crowdsourcing human-based computation”, Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, ACM, pp. 551-560.https://doi.org/10.1145/1868914.1868976
DOI

Willis, C.G., Law, E., Williams, A.C., Franzone, B.F., Bernardos, R., Bruno, L., Hopkins, C., Schorn, C., Weber, E., Park, D.S. and Davis, C.C. (2017), “CrowdCurio: an online crowdsourcing platform to facilitate climate change studies using herbarium specimens”, New Phytologist, Vol. 215 No. 1, pp. 479-488.

Yampolskiy, R.V. (2015), “On the limits of recursively self-improving AGI”, Proceedings of International Conference on Artificial General Intelligence, Springer, Cham, pp. 394-403.https://doi.org/10.1007/978-3-319-21365-1_40
DOI

Yampolskiy, R.V. and El-Barkouky, A. (2011), “Wisdom of artificial crowds algorithm for solving NP-hard problems”, International Journal of Bio-Inspired Computation, Vol. 3 No. 6, pp. 358-369.

Yang, J., Drake, T., Damianou, A. and Maarek, Y. (2018), “Leveraging crowdsourcing data for deep active learning an application: learning intents in Alexa”, Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 23-32.https://doi.org/10.1145/3178876.3186033
DOI

Yu, C., Chai, Y. and Liu, Y. (2018), “Literature review on collective intelligence: a crowd science perspective”, International Journal of Crowd Science, Vol. 2 No. 1, pp. 64-73.

Yu, H., Shen, Z., Miao, C. and An, B. (2012), “Challenges and opportunities for trust management in crowdsourcing”, Proceedings of the IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology, IEEE, Vol. 2, pp. 486-493.https://doi.org/10.1109/WI-IAT.2012.104
DOI

Zadeh, L.A. (2008), “Toward human level machine intelligence-is it achievable? The need for a paradigm shift”, IEEE Computational Intelligence Magazine, Vol. 3 No. 3.

Zanakis, S.H., Theofanides, S., Kontaratos, A.N. and Tassios, T.P. (2003), “Ancient Greeks' practices and contributions in public and entrepreneurship decision making”, Interfaces, Vol. 33 No. 6, pp. 72-88.

Zhao, X., Li, J., Han, R., Xie, B. and Ou, J. (2019), “GroundEye: a mobile crowdsourcing structure seismic response monitoring system based on smartphone”, in Health Monitoring of Structural and Biological Systems XIII, Vol. 10972, International Society for Optics and Photonics, available at: https://doi.org/10.1117/12.2514905
DOI
Zhong, J., Tang, K. and Zhou, Z.H. (2015), “Active learning from crowds with unsure option”, Proceedings of International Joint Conferences on Artificial Intelligence, pp. 1061-1068.
Publication history
Copyright
Rights and permissions

Publication history

Received: 26 March 2019
Revised: 03 June 2019
Accepted: 11 July 2019
Published: 16 August 2019
Issue date: September 2019

Copyright

© The author(s)

Rights and permissions

Morteza Moradi, Mohammad Moradi, Farhad Bayat and Adel Nadjaran Toosi. Published in International Journal of Crowd Science. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

Return