This comprehensive review examines the integration of causal reasoning in foundation models, focusing on three types of causal reasoning: commonsense causal reasoning, quantitative causal reasoning, and formal causal reasoning. Current foundation models exhibit different capabilities in processing or reasoning these three types of causal relationships with various methodologies. We propose a structured way that categorize existing methods into three tiers: data-level, where input data, such as prompts or knowledge, are crafted for causal inferences; component-level, where model internals, like structures or modules, are adjusted to yield causal insights; and model-level, where some existing foundation models have been discussed about their potential in causal inference, while several new models are designed inherently with causal reasoning capabilities. The reviews highlight the significance of these strategies in advancing AI systems towards reliable and ethical causal inference, essential for real-world applications.

Intelligent machines are knowledge systems with unique knowledge structure and function. In this paper, we discuss issues including the characteristics and forms of machine knowledge, the relationship between knowledge and human cognition, and the approach to acquire machine knowledge. These issues are of great significance to the development of artificial intelligence.