Journal Home > Volume 3 , Issue 2
International Journal of Crowd Science 2019, 3(2): 198-220 https://doi.org/10.1108/IJCS-03-2019-0012
Conceptual paper | Open Access | Issue | Published: 16 August 2019

Collective hybrid intelligence: towards a conceptual framework

Show Author's Information Morteza Moradi1,  Mohammad Moradi2,  Farhad Bayat1( ),  Adel Nadjaran Toosi3
Department of Electrical Engineering, University of Zanjan, Zanjan, Iran
Young Researchers and Elite Club, Qazvin, Islamic Republic of Iran
Faculty of Information Technology, Monash University, Melbourne, Australia
Keywords:
Crowdsourcing, Human computation, Autonomous control, Collective machine intelligence, Human–machine collaboration, Hybrid intelligence
Cite this article:
Moradi M, Moradi M, Bayat F, et al. Collective hybrid intelligence: towards a conceptual framework. International Journal of Crowd Science, 2019, 3(2): 198-220. https://doi.org/10.1108/IJCS-03-2019-0012
Download citation
EndNote(RIS)
BibTeX

401

Views

17

Downloads

Citations

13

Crossref

N/A

WoS

15

Scopus

N/A

CSCD

Abstract Full text About this article
Purpose

Human or machine, which one is more intelligent and powerful for performing computing and processing tasks? Over the years, researchers and scientists have spent significant amounts of money and effort to answer this question. Nonetheless, despite some outstanding achievements, replacing humans in the intellectual tasks is not yet a reality. Instead, to compensate for the weakness of machines in some (mostly cognitive) tasks, the idea of putting human in the loop has been introduced and widely accepted. In this paper, the notion of collective hybrid intelligence as a new computing framework and comprehensive.

Design/methodology/approach

According to the extensive acceptance and efficiency of crowdsourcing, hybrid intelligence and distributed computing concepts, the authors have come up with the (complementary) idea of collective hybrid intelligence. In this regard, besides providing a brief review of the efforts made in the related contexts, conceptual foundations and building blocks of the proposed framework are delineated. Moreover, some discussion on architectural and realization issues are presented.

Findings

The paper describes the conceptual architecture, workflow and schematic representation of a new hybrid computing concept. Moreover, by introducing three sample scenarios, its benefits, requirements, practical roadmap and architectural notes are explained.

Originality/value

The major contribution of this work is introducing the conceptual foundations to combine and integrate collective intelligence of humans and machines to achieve higher efficiency and (computing) performance. To the best of the authors’ knowledge, this the first study in which such a blessing integration is considered. Therefore, it is believed that the proposed computing concept could inspire researchers toward realizing such unprecedented possibilities in practical and theoretical contexts.


menu
Abstract
Full text
Outline
About this article

Collective hybrid intelligence: towards a conceptual framework

Show Author's information Morteza Moradi1, Mohammad Moradi2, Farhad Bayat1( ), Adel Nadjaran Toosi3
Department of Electrical Engineering, University of Zanjan, Zanjan, Iran
Young Researchers and Elite Club, Qazvin, Islamic Republic of Iran
Faculty of Information Technology, Monash University, Melbourne, Australia

Abstract

Purpose

Human or machine, which one is more intelligent and powerful for performing computing and processing tasks? Over the years, researchers and scientists have spent significant amounts of money and effort to answer this question. Nonetheless, despite some outstanding achievements, replacing humans in the intellectual tasks is not yet a reality. Instead, to compensate for the weakness of machines in some (mostly cognitive) tasks, the idea of putting human in the loop has been introduced and widely accepted. In this paper, the notion of collective hybrid intelligence as a new computing framework and comprehensive.

Design/methodology/approach

According to the extensive acceptance and efficiency of crowdsourcing, hybrid intelligence and distributed computing concepts, the authors have come up with the (complementary) idea of collective hybrid intelligence. In this regard, besides providing a brief review of the efforts made in the related contexts, conceptual foundations and building blocks of the proposed framework are delineated. Moreover, some discussion on architectural and realization issues are presented.

Findings

The paper describes the conceptual architecture, workflow and schematic representation of a new hybrid computing concept. Moreover, by introducing three sample scenarios, its benefits, requirements, practical roadmap and architectural notes are explained.

Originality/value

The major contribution of this work is introducing the conceptual foundations to combine and integrate collective intelligence of humans and machines to achieve higher efficiency and (computing) performance. To the best of the authors’ knowledge, this the first study in which such a blessing integration is considered. Therefore, it is believed that the proposed computing concept could inspire researchers toward realizing such unprecedented possibilities in practical and theoretical contexts.

Keywords: Crowdsourcing, Human computation, Autonomous control, Collective machine intelligence, Human–machine collaboration, Hybrid intelligence

References(111)

Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M. and Werthimer, D. (2002), “SETI@ home: an experiment in public-resource computing”, Communications of the Acm, Vol. 45 No. 11, pp. 56-61.
DOI Google Scholar

Atzori, L., Iera, A., Morabito, G. and Nitti, M. (2012), “The social internet of things (siot)–when social networks meet the internet of things: concept, architecture and network characterization”, Computer Networks, Vol. 56 No. 16, pp. 3594-3608.
DOI Google Scholar

Barbier, G., Zafarani, R., Gao, H., Fung, G. and Liu, H. (2012), “Maximizing benefits from crowdsourced data”, Computational and Mathematical Organization Theory, Vol. 18 No. 3, pp. 257-279.
DOI Google Scholar
Bargiela, A. and Pedrycz, W. (2016), “Granular computing”, in Handbook on Computational Intelligence, Vol. 1, Fuzzy Logic, Systems, Artificial Neural Networks, and Learning Systems, pp. 43-66.https://doi.org/10.1142/9789814675017_0002
DOI

Baruch, A., May, A. and Yu, D. (2016), “The motivations, enablers and barriers for voluntary participation in an online crowdsourcing platform”, Computers in Human Behavior, Vol. 64, pp. 923-931.
DOI Google Scholar

Bayat, F., Najafinia, S. and Aliyari, M. (2018), “Mobile robots path planning: electrostatic potential field approach”, Expert Systems with Applications, Vol. 100, pp. 68-78.
DOI Google Scholar
Beberg, A.L., Ensign, D.L., Jayachandran, G., Khaliq, S. and Pande, V.S. (2009), “Folding@ home: lessons from eight years of volunteer distributed computing”,Proceedings of International Parallel and Distributed Processing Symposium, Rome, Italy, pp. 1-8.https://doi.org/10.1109/IPDPS.2009.5160922
DOI

Bien, Z., Bang, W.C., Kim, D.Y. and Han, J.S. (2002), “Machine intelligence quotient: its measurements and applications”, Fuzzy Sets and Systems, Vol. 127 No. 1, pp. 3-16.
DOI Google Scholar

Bonabeau, E. (2009), “Decisions 2.0: the power of collective intelligence”, MIT Sloan Management Review, Article 45, Vol. 50 No. No. 2.
Google Scholar
Bonomi, F., Milito, R., Zhu, J. and Addepalli, S. (2012), “Fog computing and its role in the internet of things”, Proceedings of the first edition of the MCC workshop on Mobile Cloud Computing, ACM; pp. 13-16.https://doi.org/10.1145/2342509.2342513
DOI
Boutsis, I. and Kalogeraki, V. (2016), ““Location privacy for crowdsourcing applications”, Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, ACM, pp. 694-705.https://doi.org/10.1145/2971648.2971741
DOI

Brabham, D.C. (2008), “Crowdsourcing as a model for problem solving: an introduction and cases”, Convergence: The International Journal of Research into New Media Technologies, Vol. 14 No. 1, pp. 75-90.
DOI Google Scholar

Breazeal, C., DePalma, N., Orkin, J., Chernova, S. and Jung, M. (2013), “Crowdsourcing human-robot interaction: new methods and system evaluation in a public environment”, Journal of Human-Robot Interaction, Vol. 2 No. 1, pp. 82-111.
DOI Google Scholar

Buchanan, L. and O'Connell, A. (2006), “A brief history of decision making”, Harvard Business Review, Vol. 84 No. 1, pp. 32-40.
Google Scholar

Bundy, A. (2017), “Smart machines are not a threat to humanity”, Communications of the Acm, Vol. 60 No. 2, pp. 40-42.
DOI Google Scholar

Burnstein, E. and Berbaum, M.L. (1983), “Stages in group decision making: the decomposition of historical narratives”, Political Psychology, Vol. 4 No. 3, pp. 531-561.
DOI Google Scholar

Cassimatis, N.L. (2006), “A cognitive substrate for achieving human-level intelligence”, AI Magazine, Vol. 27 No. 2, pp. 45-45.
Google Scholar
Chang, J.C., Amershi, S. and Kamar, E. (2017), “Revolt: collaborative crowdsourcing for labeling machine learning datasets”, Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ACM, pp. 2334-2346.https://doi.org/10.1145/3025453.3026044
DOI
Chen, S.C.Y. and Shen, M.C. (2019), “The fourth industrial revolution and the development of artificial intelligence”, Contemporary Issues in International Political Economy, Palgrave Macmillan, Singapore, pp. 333-346.https://doi.org/10.1007/978-981-13-6462-4_14
DOI

Chien, A., Calder, B., Elbert, S. and Bhatia, K. (2003), “Entropia: architecture and performance of an enterprise desktop grid system”,Journal of Parallel and Distributed Computing, Vol. 63 No. 5, pp. 597-610.
DOI Google Scholar

Chiu, C.M., Liang, T.P. and Turban, E. (2014), “What can crowdsourcing do for decision support?”, Decision Support Systems, Vol. 65, pp. 40-49.
DOI Google Scholar
Copeland, B.J. (2000), The Modern History of Computing, Zalta, E.N. (Ed.), Winter 2017 Edition The Stanford Encyclopedia of Philosophy, available at: https://plato.stanford.edu/archives/win2017/entries/computing-history/
Dai, P. and Weld, D.S. (2011), “Artificial intelligence for artificial intelligence”, Proceedings of Twenty-Fifth AAAI Conference on Artificial Intelligence.

Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B. and Allahbakhsh, M. (2018), “Quality control in crowdsourcing: a survey of quality attributes, assessment techniques, and assurance actions”, ACM Computing Surveys, Article 7, Vol. 51 No. 1.
DOI Google Scholar
Decker, M. (2000), “Replacing human beings by robots. How to tackle that perspective by technology assessment”, in Vision Assessment: Shaping Technology in 21st Century Society, Springer Berlin Heidelberg, Berlin, Heidelberg, pp. 149-166.https://doi.org/10.1007/978-3-642-59702-2_7
DOI
Del Prado, G.M. (2015), “18 Artificial intelligence researchers reveal the profound changes coming to our lives”, Business Insider, available at: www.businessinsider.com/researchers-predictions-future-artificial-intelligence-2015-10 (accessed 10 September 2018).

Deng, W., Chen, R., Gao, J., Song, Y. and Xu, J. (2012), “A novel parallel hybrid intelligence optimization algorithm for a function approximation problem”, Computers and Mathematics with Applications, Vol. 63 No. 1, pp. 325-336.
DOI Google Scholar
Difallah, D.E., Demartini, G. and Cudré-Mauroux, P. (2012), “Mechanical cheat: spamming schemes and adversarial techniques on crowdsourcing platforms”, Proceedings of the First International Workshop on Crowdsourcing Web Search, pp. 26-30.https://doi.org/10.1145/2187836.2187900
DOI
Dobrev, D. (2012), A Definition of Artificial Intelligence”, arXiv preprint arXiv:1210.1568.
Dodge, S. and Karam, L. (2017), “A study and comparison of human and deep learning recognition performance under visual distortions”, Proceedings of the 26th International Conference on Computer Communication and Networks (ICCCN), IEEE, pp. 1-7.https://doi.org/10.1109/ICCCN.2017.8038465
DOI
Everitt, T. and Hutter, M. (2018), “Universal artificial intelligence”, In Foundations of Trusted Autonomy, Springer, Cham; pp. 15-46.https://doi.org/10.1007/978-3-319-64816-3_2
DOI

Fabisiak, T. and Danilecki, A. (2017), “Browser-based harnessing of voluntary computational power”, Foundations of Computing and Decision Sciences, Vol. 42 No. 1, pp. 3-42.
DOI Google Scholar

Feigenbaum, E.A. (2003), “Some challenges and grand challenges for computational intelligence”, Journal of the ACM ( Acm), Vol. 50 No. 1, pp. 32-40.
DOI Google Scholar

Feng, W., Yan, Z., Zhang, H., Zeng, K., Xiao, Y. and Hou, Y.T. (2017), “A survey on security, privacy, and trust in mobile crowdsourcing”, IEEE Internet of Things Journal, Vol. 5 No. 4, pp. 2971-2992.
DOI Google Scholar

Fleuret, F., Li, T., Dubout, C., Wampler, E.K., Yantis, S. and Geman, D. (2011), “Comparing machines and humans on a visual categorization test”, Proceedings of the National Academy of Sciences, Vol. 108 No. 43, pp. 17621-17625.
DOI Google Scholar
Folds, D.J. (2016), “Human executive control of autonomous systems: a conceptual framework”, Proceedings of IEEE International Symposium on Systems Engineering (ISSE), pp. 1-5.https://doi.org/10.1109/SysEng.2016.7753126
DOI
Gifford, C.M. (2009), “Collective machine learning: team learning and classification in multi-agent systems”, Doctoral dissertation, University of Kansas.
Guazzini, A., Vilone, D., Donati, C., Nardi, A. and Levnajić, Z. (2015), “Modeling crowdsourcing as collective problem solving”, Scientific reports, Vol. 5, Article number: 16557.https://doi.org/10.1038/srep16557
DOI

Gurkaynak, G., Yilmaz, I. and Haksever, G. (2016), “Stifling artificial intelligence: human perils”, Computer Law and Security Review, Vol. 32 No. 5, pp. 749-758.
DOI Google Scholar
Halmes, M. (2013), “Measurements of collective machine intelligence”, arXiv preprint arXiv:1306.6649.
Hernández-Orallo, J. and Minaya-Collado, N. (1998), “A formal definition of intelligence based on an intensional variant of algorithmic complexity”, Proceedings of International Symposium of Engineering of Intelligent Systems (EIS98), pp. 146-163.

Hibbard, B. (2001), “Super-intelligent machines”, ACM SIGGRAPH Computer Graphics, Vol. 35 No. 1, pp. 11-13.
DOI Google Scholar

Howe, J. (2006), “The rise of crowdsourcing”, Wired Magazine, Vol. 14 No. 6, pp. 1-4.
Google Scholar

Huang, Y., Vir Singh, P. and Srinivasan, K. (2014), “Crowdsourcing new product ideas under consumer learning”, Management Science, Vol. 60 No. 9, pp. 2138-2159.
DOI Google Scholar
Huang, F.Y., Wang, K., An, Y. and Lasecki, W.S. (2017), “Towards hybrid intelligence for robotics”, Proceedings of The 5th Edition of the Collective Intelligence Conference.

Ilyashenko, A.S., Lukashin, A.A., Zaborovsky, V.S. and Lukashin, A.A. (2017), “Algorithms for planning resource-intensive computing tasks in a hybrid supercomputer environment for simulating the characteristics of a quantum rotation sensor and performing engineering calculations”, Automatic Control and Computer Sciences, Vol. 51 No. 6, pp. 426-434.
DOI Google Scholar

Jonathan, A., Ryden, M., Oh, K., Chandra, A. and Weissman, J. (2017), “Nebula: distributed edge cloud for data intensive computing”, IEEE Transactions on Parallel and Distributed Systems, Vol. 28 No. 11, pp. 3229-3242.
DOI Google Scholar

Kajino, H., Arai, H. and Kashima, H. (2014), “Preserving worker privacy in crowdsourcing”, Data Mining and Knowledge Discovery, Vol. 28 Nos 5/6, pp. 1314-1335.
DOI Google Scholar
Kamar, E. (2016), “Directions in hybrid intelligence: complementing AI systems with human intelligence”, Proceedings of IJCAI, pp. 4070-4073.
Karimi, H.A. (Ed.) (2004), “Telegeoinformatics: Location-Based Computing and Services”, CRC Press.https://doi.org/10.1201/b12395
DOI
Kennedy, J. (2006), “Swarm intelligence”, in Handbook of Nature-Inspired and Innovative Computing, Springer, pp. 187-219.https://doi.org/10.1007/0-387-27705-6_6
DOI

Kephart, J.O. and Chess, D.M. (2003), “The vision of autonomic computing”, Computer, Vol. 1, pp. 41-50.
DOI Google Scholar

Khoi, N., Casteleyn, S., Moradi, M. and Pebesma, E. (2018), “Do monetary incentives influence users’ behavior in participatory sensing?”, Sensors, Vol. 18 No. 5, Article 1426.
DOI Google Scholar

Klumpp, M., Hesenius, M., Meyer, O., Ruiner, C. and Gruhn, V. (2019), “Production logistics and human-computer interaction—state-of-the-art, challenges and requirements for the future”, The International Journal of Advanced Manufacturing Technology, doi: 10.1007/s00170-019-03785-0.
Google Scholar
Kondo, D., Andrzejak, A. and Anderson, D.P. (2008), “On correlated availability in internet-distributed systems”, Proceedings of the 2008 9th IEEE/ACM International Conference on Grid Computing, IEEE, pp. 276-283.https://doi.org/10.1109/GRID.2008.4662809
DOI
Kube, C.R. and Zhang, H. (1992), “Collective robotic intelligence”, Proceedings of the Second International Conference on Simulation of Adaptive Behavior, pp. 460-468.
Larson, S.M. Snow, C.D. Shirts, M. and Pande, V.S. (2009), “Folding@ Home and Genome@ Home: Using distributed computing to tackle previously intractable problems in computational biology”, arXiv preprint arXiv:0901.0866.

Legg, S. and Hutter, M. (2007), “Universal intelligence: a definition of machine intelligence”, Minds and Machines, Vol. 17 No. 4, pp. 391-444.
DOI Google Scholar

Leimeister, J.M. (2010), “Collective intelligence”, Business and Information Systems Engineering, Vol. 2 No. 4, pp. 245-248.
DOI Google Scholar

Li, J., Pan, Z., Xu, J., Liang, B., Chen, Y. and Ji, W. (2018), “Quality-time-complexity universal intelligence measurement”, International Journal of Crowd Science, Vol. 2 No. 2, pp. 99-107.
DOI Google Scholar

Litman, L., Robinson, J. and Abberbock, T. (2017), “TurkPrime. com: a versatile crowdsourcing data acquisition platform for the behavioral sciences”, Behavior Research Methods, Vol. 49 No. 2, pp. 433-442.
DOI Google Scholar

Liu, J., Pacitti, E., Valduriez, P. and Mattoso, M. (2015), “A survey of data-intensive scientific workflow management”, Journal of Grid Computing, Vol. 13 No. 4, pp. 457-493.
DOI Google Scholar

Lu, H., Li, Y., Chen, M., Kim, H. and Serikawa, S. (2018), “Brain intelligence: go beyond artificial intelligence”, Mobile Networks and Applications, Vol. 23 No. 2, pp. 368-375.
DOI Google Scholar

Luo, S., Xia, H., Yoshida, T. and Wang, Z. (2009), “Toward collective intelligence of online communities: a primitive conceptual model”, Journal of Systems Science and Systems Engineering, Vol. 18 No. 2, pp. 203-221.
DOI Google Scholar

Mahoney, M.S. (1988), “The history of computing in the history of technology”, Ieee Annals of the History of Computing, Vol. 10 No. 2, pp. 113-125.
DOI Google Scholar

Makridakis, S. (2017), “The forthcoming artificial intelligence (AI) revolution: its impact on society and firms”, Futures, Vol. 90, pp. 46-60.
DOI Google Scholar

Maleszka, M. and Nguyen, N.T. (2015), “Integration computing and collective intelligence”, Expert Systems with Applications, Vol. 42 No. 1, pp. 332-340.
DOI Google Scholar
Mao, A., Kamar, E., Chen, Y., Horvitz, E., Schwamb, M.E., Lintott, C.J. and Smith, A.M. (2013), “Volunteering versus work for pay: incentives and tradeoffs in crowdsourcing”, Proceedings of the First AAAI Conference on Human Computation and Crowdsourcing.

Maoz, Z. (1990), “Framing the national interest: the manipulation of foreign policy decisions in group settings”, World Politics, Vol. 43 No. 1, pp. 77-110.
DOI Google Scholar

Miailhe, N. and Hodes, C. (2017), “The third age of artificial intelligence. Field actions science reports”, The Journal of Field Actions, (Special Issue), Vol. 17, pp. 6-11.
Google Scholar
Müller, V.C. and Bostrom, N. (2016), “Future progress in artificial intelligence: a survey of expert opinion”, In Fundamental Issues of Artificial Intelligence, Springer, Cham, pp. 555-572.https://doi.org/10.1007/978-3-319-26485-1_33
DOI

Nilsson, N.J. (2005), “Human-level artificial intelligence? Be serious!”, AI Magazine, Vol. 26 No. 4, pp. 68-68.
Google Scholar
Nowak, S. and Rüger, S. (2010), “How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation”, Proceedings of the international conference on Multimedia Information Retrieval, ACM, pp. 557-566.https://doi.org/10.1145/1743384.1743478
DOI
Nushi, B., Kamar, E. and Horvitz, E. (2018), “Towards accountable AI: hybrid human-machine analyses for characterizing system failure”, Proceedings of the Sixth AAAI Conference on Human Computation and Crowdsourcing.

Ofli, F., Meier, P., Imran, M., Castillo, C., Tuia, D., Rey, N., Briant, J., Millet, P., Reinhard, F., Parkan, M. and Joost, S. (2016), “Combining human computing and machine learning to make sense of big (aerial) data for disaster response”, Big Data, Vol. 4 No. 1, pp. 47-59.
DOI Google Scholar

Pan, Y. (2016), “Heading toward artificial intelligence 2.0”, Engineering, Vol. 2 No. 4, pp. 409-413.
DOI Google Scholar
Pedreira, M.M. and Grigoras, C. (2017), “Scalable Global Grid catalogue for LHC Run3 and beyond”, arXiv preprint arXiv:1704.05272.

Peer, E., Brandimarte, L., Samat, S. and Acquisti, A. (2017), “Beyond the Turk: alternative platforms for crowdsourcing behavioral research”, Journal of Experimental Social Psychology, Vol. 70, pp. 153-163.
DOI Google Scholar

Pilz, D. and Gewald, H. (2013), “Does money matter? Motivational factors for participation in paid-and non-profit-crowdsourcing communities”, Wirtschaftsinformatik, Vol. 37, pp. 73-82.
Google Scholar
Quinn, A.J. and Bederson, B.B. (2011), “Human computation: a survey and taxonomy of a growing field”, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, pp. 1403-1412.https://doi.org/10.1145/1978942.1979148
DOI

Rawat, A.S., Papailiopoulos, D.S., Dimakis, A.G. and Vishwanath, S. (2016), “Locality and availability in distributed storage”, IEEE Transactions on Information Theory, Vol. 62 No. 8, pp. 4481-4493.
DOI Google Scholar
Retelny, D., Robaszkiewicz, S., To, A., Lasecki, W.S., Patel, J., Rahmati, N., Doshi, T., Valentine, M. and Bernstein, M.S. (2014), “Expert crowdsourcing with flash teams”, Proceedings of the 27th annual ACM symposium on User Interface Software and Technology, ACM, pp. 75-85.https://doi.org/10.1145/2642918.2647409
DOI

Russell, S. (2017), “Artificial intelligence: the future is superintelligent”, Nature, Vol. 548 No. 7669, pp. 520-522.
DOI Google Scholar
Sadashiv, N. and Kumar, S.D. (2011), “Cluster, grid and cloud computing: a detailed comparison”,Proceedings of the 6th International Conference on Computer Science and Education (ICCSE), IEEE, pp. 477-482.https://doi.org/10.1109/ICCSE.2011.6028683
DOI

Sarathy, V. (2018), “Real world problem-solving”, Frontiers in Human Neuroscience, Vol. 12PMC6028615.
DOI Google Scholar

Schemmann, B., Herrmann, A.M., Chappin, M.M. and Heimeriks, G.J. (2016), “Crowdsourcing ideas: involving ordinary users in the ideation phase of new product development”, Research Policy, Vol. 45 No. 6, pp. 1145-1154.
DOI Google Scholar

Shi, W., Cao, J., Zhang, Q., Li, Y. and Xu, L. (2016), “Edge computing: vision and challenges”, IEEE Internet of Things Journal, Vol. 3 No. 5, pp. 637-646.
DOI Google Scholar

Singh, S.P., Nayyar, A., Kaur, H. and Singla, A. (2019), “Dynamic task scheduling using balanced VM allocation policy for fog computing platforms”, Scalable Computing: Practice and Experience, Vol. 20 No. 2, pp. 433-456.
DOI Google Scholar
Stabinger, S., Rodríguez-Sánchez, A. and Piater, J. (2016), “25 Years of CNNs: Can we compare to human abstraction capabilities?”, Proceedings of International Conference on Artificial Neural Networks, Springer, Cham, pp. 380-387.https://doi.org/10.1007/978-3-319-44781-0_45
DOI
Stahl, F., Gaber, M.M., Bramer, M. and Philip, S.Y. (2010), “Pocket data mining: towards collaborative data mining in mobile computing environments”, Proceedings of 22nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI), IEEE, Vol. 2, pp. 323-330.https://doi.org/10.1109/ICTAI.2010.118
DOI

Steinhardt, J., Valiant, G. and Charikar, M. (2016), “Avoiding imposters and delinquents: adversarial crowdsourcing and peer prediction”, In Advances in Neural Information Processing Systems, pp. 4439-4447.
Google Scholar

Vaughan, J.W. (2017), “Making better use of the crowd: how crowdsourcing can advance machine learning research”, Journal of Machine Learning Research, Vol. 18, pp. 1-46.
Google Scholar

Verhulst, S.G. (2018), “Where and when AI and CI meet: exploring the intersection of artificial and collective intelligence towards the goal of innovating how we govern”,AI and Society, Vol. 33 No. 2, pp. 293-297.
DOI Google Scholar

Vij, D. and Aggarwal, N. (2018), “Smartphone based traffic state detection using acoustic analysis and crowdsourcing”, Applied Acoustics, Vol. 138, pp. 80-91.
DOI Google Scholar

Vochin, M., Zoican, S. and Borcoci, E. (2018), “Intelligent vehicle navigation system with assistance and alerting capabilities”, Concurrency and Computation: Practice and Experience, p. e4402, available at: https://doi.org/10.1002/cpe.4402
DOI Google Scholar
Von Ahn, L. (2008), “Human computation”, Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, pp. 1-2.https://doi.org/10.1109/ICDE.2008.4497403
DOI

Wang, F.Y., Carley, K.M., Zeng, D. and Mao, W. (2007), “Social computing: from social informatics to social intelligence”, IEEE Intelligent Systems, Vol. 22 No. 2.
DOI Google Scholar
Wang, W., Johnston, B. and Williams, M.A. (2012), “Social networking for robots to share knowledge, skills and know-how”, Proceedings of International Conference on Social Robotics, Springer, Berlin, Heidelberg, pp. 418-427.https://doi.org/10.1007/978-3-642-34103-8_42
DOI

Weyer, J., Fink, R.D. and Adelt, F. (2015), “Human–machine cooperation in smart cars: an empirical investigation of the loss-of-control thesis”, Safety Science, Vol. 72, pp. 199-208.
DOI Google Scholar
Wiedermann, J. (2012), “Is there something beyond AI? Frequently emerging, but seldom answered questions about artificial Super-Intelligence”, Proceedings of the International Conference Beyond AI, pp. 76-86.
Wightman, D. (2010), “Crowdsourcing human-based computation”, Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, ACM, pp. 551-560.https://doi.org/10.1145/1868914.1868976
DOI

Willis, C.G., Law, E., Williams, A.C., Franzone, B.F., Bernardos, R., Bruno, L., Hopkins, C., Schorn, C., Weber, E., Park, D.S. and Davis, C.C. (2017), “CrowdCurio: an online crowdsourcing platform to facilitate climate change studies using herbarium specimens”, New Phytologist, Vol. 215 No. 1, pp. 479-488.
DOI Google Scholar
Yampolskiy, R.V. (2015), “On the limits of recursively self-improving AGI”, Proceedings of International Conference on Artificial General Intelligence, Springer, Cham, pp. 394-403.https://doi.org/10.1007/978-3-319-21365-1_40
DOI

Yampolskiy, R.V. and El-Barkouky, A. (2011), “Wisdom of artificial crowds algorithm for solving NP-hard problems”, International Journal of Bio-Inspired Computation, Vol. 3 No. 6, pp. 358-369.
DOI Google Scholar
Yang, J., Drake, T., Damianou, A. and Maarek, Y. (2018), “Leveraging crowdsourcing data for deep active learning an application: learning intents in Alexa”, Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 23-32.https://doi.org/10.1145/3178876.3186033
DOI

Yu, C., Chai, Y. and Liu, Y. (2018), “Literature review on collective intelligence: a crowd science perspective”, International Journal of Crowd Science, Vol. 2 No. 1, pp. 64-73.
DOI Google Scholar
Yu, H., Shen, Z., Miao, C. and An, B. (2012), “Challenges and opportunities for trust management in crowdsourcing”, Proceedings of the IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology, IEEE, Vol. 2, pp. 486-493.https://doi.org/10.1109/WI-IAT.2012.104
DOI

Zadeh, L.A. (2008), “Toward human level machine intelligence-is it achievable? The need for a paradigm shift”, IEEE Computational Intelligence Magazine, Vol. 3 No. 3.
DOI Google Scholar

Zanakis, S.H., Theofanides, S., Kontaratos, A.N. and Tassios, T.P. (2003), “Ancient Greeks' practices and contributions in public and entrepreneurship decision making”, Interfaces, Vol. 33 No. 6, pp. 72-88.
DOI Google Scholar
Zhao, X., Li, J., Han, R., Xie, B. and Ou, J. (2019), “GroundEye: a mobile crowdsourcing structure seismic response monitoring system based on smartphone”, in Health Monitoring of Structural and Biological Systems XIII, Vol. 10972, International Society for Optics and Photonics, available at: https://doi.org/10.1117/12.2514905
DOI
Zhong, J., Tang, K. and Zhou, Z.H. (2015), “Active learning from crowds with unsure option”, Proceedings of International Joint Conferences on Artificial Intelligence, pp. 1061-1068.
Publication history
Copyright
Rights and permissions

Publication history

Received: 26 March 2019
Revised: 03 June 2019
Accepted: 11 July 2019
Published: 16 August 2019
Issue date: September 2019

Copyright

© The author(s)

Rights and permissions

Morteza Moradi, Mohammad Moradi, Farhad Bayat and Adel Nadjaran Toosi. Published in International Journal of Crowd Science. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

Return