As a critical computation in numerical simulation applications such as large-scale scientific computing and industrial simulation, the solving rate of sparse linear equations directly determines the execution efficiency of computing tasks. However, sparse matrix computations are characterized by low computational intensity and high memory occupation, which results in performance bottlenecks in solving sparse linear equations. Many studies employ parallel optimization techniques to enhance the efficiency of solving sparse linear equations, but they all encounter many challenges such as low storage efficiency, load imbalance, and discontinuous memory access. Therefore, this paper first analyzes the challenges in improving the efficiency of solving sparse linear equations. Then the parallel optimization methods to improve the efficiency of solving ultra-large-scale sparse linear equations are sorted out from four key aspects, including: optimization for the sparse matrix storage format, optimization for solving ultra-large-scale sparse linear equations, optimization for the basic operator of sparse matrix computation and mainstream sparse linear solver. Finally, this work concludes with a summary and a discussion of the directions that parallel optimization research in the future will go in solving ultra-large-scale sparse linear equations.

Large models have been widely used in the field of neural language processing, information retrieving, etc. With the development of the large models, not only is the parameter scale increased, but the model architecture has also become more complex. For example, the multi-modal transformer-based model mainly has concurrent branches, which we denoted as the concurrent branch model (CBM). Many CBMs have enlarged to tens of billions of parameters, and require distributed resources to train this kind of model. Existing distributed training systems cannot fully handle this type of model architecture because there are interactions between branches. Inspired by the unbalanced resource usage of pipeline parallelism, we prefer to organize different branches with a fine-grained bidirectional pipeline schedule of communication and computation. However, improper coordination between branches leads to idle time for computation and low training efficiency. In this paper, we present Flexpipe, a pipeline engine for c3oncurrent-branch models. We first introduce a branch-aware pipeline parallelism (BAPP) to make full use of the concurrent characteristic of the model architecture. Then, based on a multi-branch pipeline simulator, we propose an adaptive interaction coordinator, which facilitates the low-overhead branch interactions during the distributed model training. We evaluate our approach on popular concurrent branch models combined with modern training systems. Compared with the Chimera, the experiential results show that our method improves the end-to-end training throughput by 20% on average.

As the computational demands driven by large model technologies continue to grow rapidly, leveraging GPU hardware to expedite parallel training processes has emerged as a commonly-used strategy. When computational resources within a single cluster are insufficient for large-model training, the hybrid utilization of heterogeneous acceleration hardware has emerged as a promising technical solution. The utilization of heterogeneous acceleration hardware and scheduling of diverse cloud resources have become a focal point of considerable interest. However, these computing resources are often geographically distributed. Due to the lack of awareness of heterogeneous devices and network topologies, existing parallel training frameworks struggle to leverage mixed GPU resources across constrained networks effectively. To boost the computing capability of the connected heterogeneous clusters, we propose HGTrainer, an optimizer designed to plan heterogeneous parallel strategies across distributed clusters for large model training. HGTrainer can adaptively saturate heterogeneous clusters because of the expanded tunable parallelism space for heterogeneous accelerators, with the awareness of relatively lower inter-cluster bandwidth. To achieve this goal, we formulate the model partitioning problem among heterogeneous hardware and introduce a hierarchical searching algorithm to solve the optimization problem. Besides, a mixed-precision pipeline method is used to reduce the cost of inter-cluster communications. We evaluate HGTrainer on heterogeneous connected clusters with popular large language models. The experimental result shows that HGTrainer effectively improves 1.49× training throughput on average for the mixed heterogeneous cluster compared with the state-of-the-art Metis.

In today’s rapidly evolving internet landscape, prominent companies across various industries face increasingly complex business operations, leading to significant cluster-scale growth. However, this growth brings about challenges in cluster management and the inefficient utilization of vast amounts of data due to its low value density. This paper, based on the large-scale cluster virtualization and monitoring system of the data center of the Bureau of Geophysical Prospecting (BGP), utilizes time series data of host resources from the monitoring system’s time series database to propose a multivariate multi-step time series forecasting model, MUL-CNN-BiGRU-Attention, for forecasting CPU load on virtual cluster hosts. The model undergoes extensive offline training using a large volume of time series data, followed by deployment using TensorFlow Serving. Recent small-batch data are employed for fine-tuning model parameters to better adapt to current data patterns. Comparative experiments are conducted between the proposed model and other baseline models, demonstrating notable improvements in Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and R² metrics by up to 35.2%, 56.1%, 32.5%, and 10.3%, respectively. Additionally, ablation experiments are designed to investigate the impact of different factors on the performance of the forecasting model, providing valuable insights for parameter optimization based on experimental results.

The radiation damage effect of key structural materials is one of the main research subjects of the numerical reactor. From the perspective of experimental safety and feasibility, Molecular Dynamics (MD) in the materials field is an ideal method for simulating the radiation damage of structural materials. The Crystal-MD represents a massive parallel MD simulation software based on the key material characteristics of reactors. Compared with the Large-scale Atomic/Molecurlar Massively Parallel Simulator (LAMMPS) and ITAP Molecular Dynamics (IMD) software, the Crystal-MD reduces the memory required for software operation to a certain extent, but it is very time-consuming. Moreover, the calculation results of the Crystal-MD have large deviations, and there are also some problems, such as memory limitation and frequent communication during its migration and optimization. In this paper, in order to solve the above problems, the memory access mode of the Crystal-MD software is studied. Based on the memory access mode, a memory access optimization strategy is proposed for a unique architecture of China’s supercomputer Sunway TaihuLight. The proposed optimization strategy is verified by the experiments, and experimental results show that the running speed of the Crystal-MD is increased significantly by using the proposed optimization strategy.

With the development of Chinese international trade, real-time processing systems based on ship trajectory have been used to cluster trajectory in real-time, so that the hot zone information of a sea ship can be discovered in real-time. This technology has great research value for the future planning of maritime traffic. However, ship navigation characteristics cannot be found in real-time with a ship Automatic Identification System (AIS) positioning system, and the clustering effect based on the density grid fixed-time-interval algorithm cannot resolve the shortcomings of real-time clustering. This study proposes an adaptive time interval clustering algorithm based on density grid (called DAC-Stream). This algorithm can perform adaptive time-interval clustering according to the size of the real-time ship trajectory data stream, so that a ship’s hot zone information can be found efficiently and in real-time. Experimental results show that the DAC-Stream algorithm improves the clustering effect and accelerates data processing compared with the fixed-time-interval clustering algorithm based on density grid (called DC-Stream).

MapReduce is currently the most popular programming model for big data processing, and Hadoop is a well-known MapReduce implementation platform. However, Hadoop jobs suffer from imbalanced workloads during the reduce phase and inefficiently utilize the available computing and network resources. In some cases, these problems lead to serious performance degradation in MapReduce jobs. To resolve these problems, in this paper, we propose two algorithms, the Locality-Based Balanced Schedule (LBBS) and Overlapping-Based Resource Utilization (OBRU), that optimize the Locality-Enhanced Load Balance (LELB) and the Map, Local reduce, Shuffle, and final Reduce (MLSR) phases. The LBBS collects partition information from input data during the map phase and generates balanced schedule plans for the reduce phase. OBRU is responsible for using computing and network resources efficiently by overlapping the local reduce, shuffle, and final reduce phases. Experimental results show that the LBBS and OBRU algorithms yield significant improvements in load balancing. When LBBS and OBRU are applied, job performance increases by 15% from that of models using LELB and MLSR.

Kinetic Monte Carlo (KMC) is a widely used method for studying the evolution of materials at themicrocosmic level. At present, while there are many simulation software programs based on this algorithm, most focus on the verification of a certain phenomenon and have no analog-scale requirement, so many are serial in nature. The dynamic Monte Carlo algorithm is implemented using a parallel framework called SPPARKS, but it does not support the Embedded Atom Method (EAM) potential, which is commonly used in the dynamic simulation of metal materials. Metal material — the preferred material for most containers and components — plays an important role in many fields, including construction engineering and transportation. In this paper, we propose and describe the development of a parallel software program called Crystal-KMC, which is specifically used to simulate the lattice dynamics of metallic materials. This software uses MPI to achieve a parallel multiprocessing mode, which avoid the limitations of serial software in the analog scale. Finally, we describe the use of the parallel-KMC simulation software Crystal-KMC in simulating the diffusion of vacancies in iron, and analyze the experimental results. In addition, we tested the performance of Crystal-KMC in “meta -Era” supercomputing clusters, and the results show the Crystal-KMC parallel software to have good parallel speedup and scalability.

With the rise of various cloud services, the problem of redundant data is more prominent in the cloud storage systems. How to assign a set of documents to a distributed file system, which can not only reduce storage space, but also ensure the access efficiency as much as possible, is an urgent problem which needs to be solved. Space-efficiency mainly uses data de-duplication technologies, while access-efficiency requires gathering the files with high similarity on a server. Based on the study of other data de-duplication technologies, especially the Similarity-Aware Partitioning (SAP) algorithm, this paper proposes the Frequency and Similarity-Aware Partitioning (FSAP) algorithm for cloud storage. The FSAP algorithm is a more reasonable data partitioning algorithm than the SAP algorithm. Meanwhile, this paper proposes the Space-Time Utility Maximization Model (STUMM), which is useful in balancing the relationship between space-efficiency and access-efficiency. Finally, this paper uses 100 web files downloaded from CNN for testing, and the results show that, relative to using the algorithms associated with the SAP algorithm (including the SAP-Space-Delta algorithm and the SAP-Space-Dedup algorithm), the FSAP algorithm based on STUMM reaches higher compression ratio and a more balanced distribution of data blocks.