In addition to a physical comprehension of the world, humans possess a high social intelligence—the intelligence that senses social events, infers the goals and intents of others, and facilitates social interaction. Notably, humans are distinguished from their closest primate cousins by their social cognitive skills as opposed to their physical counterparts. We believe that artificial social intelligence (ASI) will play a crucial role in shaping the future of artificial intelligence (AI). This article begins with a review of ASI from a cognitive science standpoint, including social perception, theory of mind (ToM), and social interaction. Next, we examine the recently-emerged computational counterpart in the AI community. Finally, we provide an in-depth discussion on topics related to ASI.
T. Ziemke and S. Thellman, Do we really want AI to be human-like? Sci. Robot., vol. 7, no. 68, p. eadd0641, 2022.
A. M. Turing, Computing machinery and intelligence, Mind, vol. 59, pp. 443–460, 1950.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., Human-level control through deep reinforcement learning, Nature, vol. 518, no. 7540, pp. 529–533, 2015.
J. B. Tenenbaum, C. Kemp, T. L. Griffiths, and N. D. Goodman, How to grow a mind: Statistics, structure, and abstraction, Science, vol. 331, no. 6022, pp. 1279–1285, 2011.
B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum, Human-level concept learning through probabilistic program induction, Science, vol. 350, no. 6266, pp. 1332–1338, 2015.
B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman, Building machines that learn and think like people, Behav. Brain Sci., vol. 40, p. e253, 2017.
Y. Zhu, T. Gao, L. Fan, S. Huang, M. Edmonds, H. Liu, F. Gao, C. Zhang, S. Qi, Y. N. Wu, et al., Dark, beyond deep: A paradigm shift to cognitive AI with humanlike common sense, Engineering, vol. 6, no. 3, pp. 310–345, 2020.
T. Shu, Y. Peng, S. C. Zhu, and H. Lu, A unified psychological space for human perception of physical and social events, Cogn. Psychol., vol. 128, p. 101398, 2021.
E. S. Spelke and K. D. Kinzler, Core knowledge, Dev. Sci., vol. 10, no. 1, pp. 89–96, 2007.
J. R. Kubricht, K. J. Holyoak, and H. Lu, Intuitive physics: Current research and controversies, Trends Cogn. Sci., vol. 21, no. 10, pp. 749–759, 2017.
A. Newell, Physical symbol systems, Cogn. Sci., vol. 4, no. 2, pp. 135–183, 1980.
I. Biederman, R. J. Mezzanotte, and J. C. Rabinowitz, Scene perception: Detecting and judging objects undergoing relational violations, Cogn. Psychol., vol. 14, no. 2, pp. 143–177, 1982.
P. W. Battaglia, J. B. Hamrick, and J. B. Tenenbaum, Simulation as an engine of physical scene understanding, Proc. Natl. Acad. Sci., vol. 110, no. 45, pp. 18327–18332, 2013.
T. D. Ullman, E. Spelke, P. Battaglia, and J. B. Tenenbaum, Mind games: Game engines as an architecture for intuitive physics, Trends Cogn. Sci., vol. 21, no. 9, pp. 649–665, 2017.
T. Ye, S. Qi, J. Kubricht, Y. Zhu, H. Lu, and S. C. Zhu, The Martian: Examining human physical judgments across virtual gravity fields, IEEE Trans. Vis. Comput. Graph., vol. 23, no. 4, pp. 1399–1408, 2017.
L. S. Piloto, A. Weinstein, P. Battaglia, and M. Botvinick, Intuitive physics learning in a deep-learning model inspired by developmental psychology, Nat. Hum. Behav., vol. 6, no. 9, pp. 1257–1267, 2022.
C. Jiang, S. Qi, Y. Zhu, S. Huang, J. Lin, L. F. Yu, D. Terzopoulos, and S. C. Zhu, Configurable 3D scene synthesis and 2D image rendering with per-pixel ground truth using stochastic grammars, Int. J. Comput. Vis., vol. 126, no. 9, pp. 920–941, 2018.
B. Zheng, Y. Zhao, J. Yu, K. Ikeuchi, and S. C. Zhu, Scene understanding by reasoning stability and safety, Int. J. Comput. Vis., vol. 112, no. 2, pp. 221–238, 2015.
Z. Zhang, Z. Jiao, W. Wang, Y. Zhu, S. C. Zhu, and H. Liu, Understanding physical effects for effective tool-use, IEEE Robot. Automat. Lett., vol. 7, no. 4, pp. 9469–9476, 2022.
M. Han, Z. Zhang, Z. Jiao, X. Xie, Y. Zhu, S. C. Zhu, and H. Liu, Scene reconstruction with functional objects for robot autonomy, Int. J. Comput. Vis., vol. 130, no. 12, pp. 2940–2961, 2022.
T. L. Griffiths and J. B. Tenenbaum, Theory-based causal induction, Psychological Review, vol. 116, no. 4, pp. 661–716, 2009.
M. Edmonds, F. Gao, H. Liu, X. Xie, S. Qi, B. Rothrock, Y. Zhu, Y. N. Wu, H. Lu, and S. C. Zhu, A tale of two explanations: Enhancing human trust by explaining robot behavior, Sci. Robot., vol. 4, no. 37, p. eaay4663, 2019.
B. Falkenhainer, K. D. Forbus, and D. Gentner, The structure-mapping engine: Algorithm and examples, Artif. Intell., vol. 41, no. 1, pp. 1–63, 1989.
P. N. Johnson-Laird, Mental models and human reasoning, Proc. Natl. Acad. Sci., vol. 107, no. 43, pp. 18243–18250, 2010.
A. Hafri and C. Firestone, The perception of relations, Trends Cogn. Sci., vol. 25, no. 6, pp. 475–492, 2021.
E. Herrmann, J. Call, M. V. Hernández-Lloreda, B. Hare, and M. Tomasello, Humans have evolved specialized skills of social cognition: The cultural intelligence hypothesis, Science, vol. 317, no. 5843, pp. 1360–1366, 2007.
E. L. Thorndike, Intelligence and its uses, Harper’s Magazine, vol. 140, pp. 227–235, 1920.
J. Williams, S. M. Fiore, and F. Jentsch, Supporting artificial social intelligence with theory of mind, Front. Artif. Intell., vol. 5, p. 750763, 2022.
D. Silvera, M. Martinussen, and T. I. Dahl, The tromsø social intelligence scale, a self-report measure of social intelligence, Scand. J. Psychol., vol. 42, no. 4, pp. 313–319, 2001.
K. Frankish, Dual-process and dual-system theories of reasoning, Philosophy Compass, vol. 5, no. 10, pp. 914–926, 2010.
B. J. Scholl and P. D. Tremoulet, Perceptual causality and animacy, Trends Cogn. Sci., vol. 4, no. 8, pp. 299–309, 2000.
F. Heider and M. Simmel, An experimental study of apparent behavior, Amer. J. Psychol., vol. 57, no. 2, pp. 243–259, 1944.
D. H. Rakison and D. Poulin-Dubois, Developmental origin of the animate–inanimate distinction, Psychol. Bull., vol. 127, no. 2, pp. 209–228, 2001.
H. M. Wellman and D. Estes, Early understanding of mental entities: A reexamination of childhood realism, Child Dev., vol. 57, no. 4, pp. 910–923, 1986.
D. S. Berry, S. J. Misovich, K. J. Kean, and R. M. Baron, Effects of disruption of structure and motion on perceptions of social causality, Pers. Soc. Psychol. Bull., vol. 18, no. 2, pp. 237–244, 1992.
Y. Luo, L. Kaufman, and R. Baillargeon, Young infants’ reasoning about physical events involving inert and self-propelled objects, Cogn. Psychol., vol. 58, no. 4, pp. 441–486, 2009.
G. Gergely, Z. Nádasdy, G. Csibra, and S. Bíró, Taking the intentional stance at 12 months of age, Cognition, vol. 56, no. 2, pp. 165–193, 1995.
G. Csibra, G. Gergely, S. Bíró, O. Koós, and M. Brockbank, Goal attribution without agency cues: The perception of ‘pure reason’ in infancy, Cognition, vol. 72, no. 3, pp. 237–267, 1999.
T. Gao, G. E. Newman, and B. J. Scholl, The psychophysics of chasing: A case study in the perception of animacy, Cogn. Psychol., vol. 59, no. 2, pp. 154–179, 2009.
T. Gao, G. McCarthy, and B. J. Scholl, The wolfpack effect: Perception of animacy irresistibly influences interactive behavior, Psychol. Sci., vol. 21, no. 12, pp. 1845–1853, 2010.
T. Gao and B. J. Scholl, Chasing vs. stalking: Interrupting the perception of animacy, J. Exp. Psychol. :Hum. Percept. Perform., vol. 37, no. 3, pp. 669–684, 2011.
B. van Buren, T. Gao, and B. J. Scholl, What are the underlying units of perceived animacy? Chasing detection is intrinsically object-based, Psychon. Bull. Rev., vol. 24, no. 5, pp. 1604–1610, 2017.
D. Premack and G. Woodruff, Does the chimpanzee have a theory of mind? Behav. Brain Sci., vol. 1, no. 4, pp. 515–526, 1978.
T. Rusch, S. Steixner-Kumar, P. Doshi, M. Spezio, and J. Gläscher, Theory of mind and decision science: Towards a typology of tasks and computational models, Neuropsychologia, vol. 146, p. 107488, 2020.
Y. Barnes-Holmes, L. McHugh, and D. Barnes-Holmes, Perspective-taking and theory of mind: A relational frame account, Behav. Anal. Today, vol. 5, no. 1, pp. 15–25, 2004.
R. Fjelland, Why general artificial intelligence will not be realized, Humanit. Soc. Sci. Commun., vol. 7, p. 10, 2020.
H. Wimmer and J. Perner, Beliefs about beliefs: Representation and constraining function of wrong beliefs in young children’s understanding of deception, Cognition, vol. 13, no. 1, pp. 103–128, 1983.
S. Baron-Cohen, A. M. Leslie, and U. Frith, Does the autistic child have a “theory of mind”? Cognition, vol. 21, no. 1, pp. 37–46, 1985.
R. Saxe, S. Carey, and N. Kanwisher, Understanding other minds: Linking developmental psychology and functional neuroimaging, Annu. Rev. Psychol., vol. 55, pp. 87–124, 2004.
C. Langley, B. I. Cirstea, F. Cuzzolin, and B. J. Sahakian, Theory of mind and preference learning at the interface of cognitive science, neuroscience, and AI: A review, Front. Artif. Intell., vol. 5, p. 778852, 2022.
C. Westby and L. Robinson, A developmental perspective for promoting theory of mind, Top. Lang. Disord., vol. 34, no. 4, pp. 362–382, 2014.
J. Perner and B. Lang, Development of theory of mind and executive control, Trends Cogn. Sci., vol. 3, no. 9, pp. 337–344, 1999.
D. A. Baldwin and J. A. Baird, Discerning intentions in dynamic human action, Trends Cogn. Sci., vol. 5, no. 4, pp. 171–178, 2001.
A. L. Woodward, Infants selectively encode the goal object of an actor’s reach, Cognition, vol. 69, no. 1, pp. 1–34, 1998.
D. A. Baldwin, J. A. Baird, M. M. Saylor, and M. A. Clark, Infants parse dynamic action, Child Dev., vol. 72, no. 3, pp. 708–717, 2001.
M. Tomasello, M. Carpenter, J. Call, T. Behne, and H. Moll, Understanding and sharing intentions: The origins of cultural cognition, Behav. Brain Sci., vol. 28, no. 5, pp. 675–691, 2005.
A. N. Meltzoff, Understanding the intentions of others: Re-enactment of intended acts by 18-month-old children, Dev. Psychol., vol. 31, no. 5, pp. 838–850, 1995.
G. Gergely, H. Bekkering, and I. Király, Rational imitation in preverbal infants, Nature, vol. 415, no. 6873, pp. 755–755, 2002.
A. L. Woodward, J. A. Sommerville, S. Gerson, A. M. E. Henderson, and J. Buresh, The emergence of intention attribution in infancy, Psychol. Learn. Motiv., vol. 51, pp. 187–222, 2009.
F. Warneken and M. Tomasello, Altruistic helping in human infants and young chimpanzees, Science, vol. 311, no. 5765, pp. 1301–1303, 2006.
J. P. Roiser and B. J. Sahakian, Hot and cold cognition in depression, CNS Spectr., vol. 18, no. 3, pp. 139–149, 2013.
A. Gopnik and H. M. Wellman, Why the child’s theory of mind really is a theory, Mind & Language, vol. 7, no. 1-2, pp. 145–171, 1992.
R. M. Gordon, Folk psychology as simulation, Mind & Language, vol. 1, no. 2, pp. 158–171, 1986.
N. J. Emery and N. S. Clayton, Comparative social cognition, Annu. Rev. Psychol., vol. 60, pp. 87–113, 2009.
S. M. Schaafsma, D. W. Pfaff, R. P. Spunt, and R. Adolphs, Deconstructing and reconstructing theory of mind, Trends Cogn. Sci., vol. 19, no. 2, pp. 65–72, 2015.
N. J. Emery, The eyes have it: The neuroethology, function and evolution of social gaze, Neurosci. Biobehav. Rev., vol. 24, no. 6, pp. 581–604, 2000.
L. Fan, W. Wang, S. C. Zhu, X. Tang, and S. Huang, Understanding human gaze communication by spatio-temporal graph reasoning, in Proc, 2019 IEEE/CVF International Conference on Computer Vision, no. ICCV, p. 5723, 5732.
H. Admoni and B. Scassellati, Social eye gaze in human-robot interaction: A review, J. Hum. -Robot Int., vol. 6, no. 1, pp. 25–63, 2017.
C. Moore and V. Corkum, Social understanding at the end of the first year of life, Dev. Rev., vol. 14, no. 4, pp. 349–372, 1994.
I. Brinck, The pragmatics of imperative and declarative pointing, Cogn. Sci. Quart., vol. 3, no. 4, pp. 429–446, 2004.
E. Bates, L. Camaioni, and V. Volterra, The acquisition of performatives prior to speech, Merrill-Palmer Quart., vol. 21, no. 3, pp. 205–226, 1975.
F. Rossano, J. Terwilliger, A. Bangerter, E. Genty, R. Heesen, and K. Zuberbühler, How 2- and 4-year-old children coordinate social interactions with peers, Phil. Trans. Roy. Soc. B, vol. 377, no. 1859, p. 20210100, 2022.
G. Pezzulo, The “interaction engine”: A common pragmatic competence across linguistic and nonlinguistic interactions, IEEE Trans. Autonom. Mental Dev., vol. 4, no. 2, pp. 105–123, 2012.
A. Cichocki and A. P. Kuleshov, Future trends for human-AI collaboration: A comprehensive taxonomy of AI/AGI using multiple intelligences and learning styles, Comput. Intell. Neurosci., vol. 2021, p. 8893795, 2021.
T. Shu, Y. Peng, L. Fan, H. Lu, and S. C. Zhu, Perception of human interaction based on motion trajectories: From aerial videos to decontextualized animations, Top. Cogn. Sci., vol. 10, no. 1, pp. 225–241, 2018.
T. Gao, C. L. Baker, N. Tang, H. Xu, and J. B. Tenenbaum, The cognitive architecture of perceived animacy: Intention, attention, and memory, Cogn. Sci., vol. 43, no. 8, p. e12775, 2019.
D. Xie, T. Shu, S. Todorovic, and S. C. Zhu, Learning and inferring “dark matter” and predicting human intents and trajectories in videos, IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 7, pp. 1639–1652, 2018.
W. Yoshida, R. J. Dolan, and K. J. Friston, Game theory of mind, PLoS Comput. Biol., vol. 4, no. 12, p. e1000254, 2008.
S. V. Albrecht and P. Stone, Autonomous agents modelling other agents: A comprehensive survey and open problems, Artif. Intell., vol. 258, pp. 66–95, 2018.
S. Arora and P. Doshi, A survey of inverse reinforcement learning: Challenges, methods and progress, Artif. Intell., vol. 297, p. 103500, 2021.
B. Arslan, N. A. Taatgen, and R. Verbrugge, Five-year-olds’ systematic errors in second-order false belief tasks are due to first-order theory of mind strategy selection: A computational modeling study, Front. Psychol., vol. 8, p. 275, 2017.
Y. Zeng, Y. Zhao, T. Zhang, D. Zhao, F. Zhao, and E. Lu, A brain-inspired model of theory of mind, Front. Neurorobot., vol. 14, p. 60, 2020.
C. L. Baker, J. Jara-Ettinger, R. Saxe, and J. B. Tenenbaum, Rational quantitative attribution of beliefs, desires and percepts in human mentalizing, Nat. Hum. Behav., vol. 1, no. 4, p. 0064, 2017.
J. Jara-Ettinger, Theory of mind as inverse reinforcement learning, Curr. Opin. Behav. Sci., vol. 29, pp. 105–110, 2019.
L. Yuan, X. Gao, Z. Zheng, M. Edmonds, Y. N. Wu, F. Rossano, H. Lu, Y. Zhu, and S. C. Zhu, In situ bidirectional human-robot value alignment, Sci. Robot., vol. 7, no. 68, p. eabm4183, 2022.
H. de Weerd, R. Verbrugge, and B. Verheij, Negotiating with other minds: The role of recursive theory of mind in negotiation with incomplete information, Auton. Agents Multi-Agent Syst., vol. 31, no. 2, pp. 250–287, 2017.
H. de Weerd, D. Diepgrond, and R. Verbrugge, Estimating the use of higher-order theory of mind using computational agents, B. E. J. Theor. Econ., vol. 18, no. 2, p. 20160184, 2018.
A. Kanwal, W. M. Qazi, M. A. Altaf, A. Athar, M. Hussain, S. T. S. Bukhari, and A. T. Apasiba, A step towards the development of socio-cognitive agent, Lahore Garrison Univ. Res. J. Comput. Sci. Informat. Technol., vol. 4, no. 3, pp. 23–38, 2020.
A. Panella and P. Gmytrasiewicz, Interactive POMDPs with finite-state models of other agents, Auton. Agents Multi-Agent Syst., vol. 31, no. 4, pp. 861–904, 2017.
M. Zhao, N. Tang, A. L. Dahmani, Y. Zhu, F. Rossano, and T. Gao, Sharing rewards undermines coordinated hunting, J. Comput. Biol., vol. 29, no. 9, pp. 1022–1030, 2022.
J. Pöppel, S. Kahl, and S. Kopp, Resonating minds-emergent collaboration through hierarchical active inference, Cogn. Comput., vol. 14, no. 2, pp. 581–601, 2022.
O. Nocentini, L. Fiorini, G. Acerbi, A. Sorrentino, G. Mancioppi, and F. Cavallo, A survey of behavioral models for social robots, Robotics, vol. 8, no. 3, p. 54, 2019.
T. J. Wiltshire, S. F. Warta, D. Barber, and S. M. Fiore, Enabling robotic social intelligence by engineering human social-cognitive mechanisms, Cogn. Syst. Res., vol. 43, pp. 190–207, 2017.
A. Lieto, M. Bhatt, A. Oltramari, and D. Vernon, The role of cognitive architectures in general artificial intelligence, Cogn. Syst. Res., vol. 48, pp. 1–3, 2018.
J. R. Anderson, C. Lebiere, M. Lovett, and L. Reder, ACT-R: A higher-level account of processing capacity, Behav. Brain Sci., vol. 21, no. 6, pp. 831–832, 1998.
J. R. Anderson, D. Bothell, M. D. Byrne, S. Douglass, C. Lebiere, and Y. Qin, An integrated theory of the mind, Psychol. Rev., vol. 111, no. 4, pp. 1036–1060, 2004.
C. Breazeal, J. Gray, and M. Berlin, An embodied cognition approach to mindreading skills for socially intelligent robots, Int. J. Robot. Res., vol. 28, no. 5, pp. 656–680, 2009.
W. G. Kennedy, M. D. Bugajska, A. M. Harrison, and J. G. Trafton, “Like-me” simulation as an effective and cognitively plausible basis for social robotics, Int. J. Soc. Robot., vol. 1, no. 2, pp. 181–194, 2009.
C. Moulin-Frier, T. Fischer, M. Petit, G. Pointeau, J. Y. Puigbo, U. Pattacini, S. C. Low, D. Camilleri, P. Nguyen, M. Hoffmann, et al., Dac-H3: A proactive robot cognitive architecture to acquire and express knowledge about the world and the self, IEEE Trans. Cogn. Dev. Syst., vol. 10, no. 4, pp. 1005–1022, 2018.
A. M. Franchi, F. Mutti, and G. Gini, From learning to new goal generation in a bioinspired robotic setup, Adv. Robot., vol. 30, no. 11-12, pp. 795–805, 2016.
N. Bard, J. N. Foerster, S. Chandar, N. Burch, M. Lanctot, H. F. Song, E. Parisotto, V. Dumoulin, S. Moitra, E. Hughes, et al., The Hanabi challenge: A new frontier for AI research, Artif. Intell., vol. 280, p. 103216, 2020.
H. Shevlin, K. Vold, M. Crosby, and M. Halina, The limits of machine intelligence, EMBO Rep., vol. 20, no. 10, p. e49177, 2019.
M. Crosby, B. Beyret, and M. Halina, The animal-AI Olympics, Nat. Mach. Intell., vol. 1, no. 5, pp. 257–257, 2019.
The authors would like to thank Prof. Tao Gao (UCLA) for brainstorming while the authors were with UCLA, Miss Zhen Chen (BIGAI) and Miss Qing Lei (PKU) for making the nice figures, and two anonymous reviews for constructive feedback. This work was supported in part by the National Key R&D Program of China (No. 2022ZD0114900) and the Beijing Nova Program.
The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).