The reliability of high-performance computing (HPC) is essential for program execution stability. However, as the hardware fault rate constantly increases, fault-tolerance techniques such as Checkpoint/Restart (C/R) introduce significant system overhead. This paper proposes Program Error Resilience-Aware Checkpointing Mechanism (ResCheckpointer) to mitigate the overhead of the C/R mechanism. The primary motivation of ResCheckpointer is that we observe that crash proneness (i.e., the probability of the program crashing after fault occurrence) varies significantly among inter- and intra-HPC programs, which prompts us to flexibly adjust checkpoint intervals for further C/R overhead optimization. Specifically, we first construct the graph neural network (GNN) based learning paradigms to excavate the complex error propagation and effect mechanisms hidden within the HPC program’s execution flow, and propose Crash-Predictor for efficiently predicting programs’ crash proneness. Based on this, we build ResCheckpointer, which equips an intelligent checkpoint interval setting strategy for HPC programs, i.e., denser for the crash proneness stage while sparser for the error resilience stage. Experimental results show that ResCheckpointer can achieve up to 55.37% C/R cost reduction compared with the baseline C/R mechanism.

Stream processing applications continuously process large amounts of online streaming data in real time or near real time. They have strict latency constraints. However, the continuous processing makes them vulnerable to any failures, and the recoveries may slow down the entire processing pipeline and break latency constraints. The upstream backup scheme is one of the most widely applied fault-tolerant schemes for stream processing systems. It introduces complex backup dependencies to tasks, which increases the difficulty of controlling recovery latencies. Moreover, when dependent tasks are located on the same processor, they fail at the same time in processor-level failures, bringing extra recovery latencies that increase the impacts of failures. This paper studies the relationship between the task allocation and the recovery latency of a stream processing application. We present a correlated failure effect model to describe the recovery latency of a stream topology in processor-level failures under a task allocation plan. We introduce a recovery-latency aware task allocation problem (RTAP) that seeks task allocation plans for stream topologies that will achieve guaranteed recovery latencies. We discuss the difference between RTAP and classic task allocation problems and present a heuristic algorithm with a computational complexity of O(n log² n) to solve the problem. Extensive experiments were conducted to verify the correctness and effectiveness of our approach. It improves the resource usage by 15%–20% on average.