Fully automated processor design has recently gained significant popularity due to its fast convergence speed and reduced human costs. However, automated design remains challenging in processor correctness and performance guarantee. In this article, we introduce a series of processor auto-design methods based on a data-driven method, Binary Speculative Diagram (BSD), emphasizing how they guarantee design correctness and improve the auto-designed processor performance. Auto-designed by BSD, QiMeng-CPU-v1, an industrial-scale RISC-V CPU, achieves up to 99.99999999999% accuracy. Auto-designed by State-BSD, QiMeng-CPU-v2 is comparable to ARM Cortex A53 (2010s CPU), a human-designed superscalar processor. Finally, we discuss potential future directions for extending and improving the proposed design methods toward more generalized automated processor architectures.

In this paper, we present a comprehensive overview of artificial intelligence (AI) computing systems for large language models (LLMs) training. The rapid advancement of LLMs in recent years, coupled with the widespread adoption of algorithms and applications such as BERT, ChatGPT, and DeepSeek, has sparked significant interest in this field. We classify LLMs into encoder-only, encoder-decoder, and decoder-only models, and briefly analyze their training and inference processes to emphasize their substantial need for computational resources. These operations depend heavily on AI-specific accelerators like GPUs (graphics processing units), TPUs (tensor processing units), and MLUs (machine learning units). However, as the gap widens between the increasing complexity of LLMs and the current capabilities of accelerators, it becomes essential to adopt heterogeneous computing systems optimized for distributed environments to manage the growing computational and memory requirements of LLMs. We delve into the execution and scheduling of LLM algorithms, underlining the critical role of distributed computing strategies, memory management enhancements, and boosting computational efficiency. This paper clarifies the complex relationship between algorithm design, hardware infrastructure, and software optimization, and provides an in-depth understanding of both the software and hardware infrastructure supporting LLMs training, offering insights into the challenges and potential avenues for future development and deployment.