Climate simulation is very challenging that it involves a large number of interacting physical processes. Community earth system model (CESM) extensively applied to predict regional and global climate, is a state-of-art open source coupled climate system. CESM application requires great amount of computation, and future ultrahigh-resolution climate simulations demand even larger scale parallelism. In recent years, the emergence of ARM-based HPC clusters have provided a novel alternative to host these cyber-physical systems. Scalability and power efficiency are two critical issues for traditional HPC (high performance computing) platforms. Compared with traditional X86 platforms, ARM-based processors provide higher memory bandwidth and more cores per chip, which can potentially benefit the application scalability. In this work, we successfully port CESM to Huawei Kunpeng platform based on ARM architecture. Based on the runtime data of CESM, a customized C/Fortran compiler is proposed and the process scheduling algorithm is improved. Extensive experiments have been conducted on Huawei Kunpeng platform and Intel Xeon platform. Results illustrate that optimized CESM instance on Huawei Kunpeng platform has notable performance improvement, 31.78%~42.93% overall, and better scalability, in spite of relatively lower single core performance.

Stream processing applications continuously process large amounts of online streaming data in real time or near real time. They have strict latency constraints. However, the continuous processing makes them vulnerable to any failures, and the recoveries may slow down the entire processing pipeline and break latency constraints. The upstream backup scheme is one of the most widely applied fault-tolerant schemes for stream processing systems. It introduces complex backup dependencies to tasks, which increases the difficulty of controlling recovery latencies. Moreover, when dependent tasks are located on the same processor, they fail at the same time in processor-level failures, bringing extra recovery latencies that increase the impacts of failures. This paper studies the relationship between the task allocation and the recovery latency of a stream processing application. We present a correlated failure effect model to describe the recovery latency of a stream topology in processor-level failures under a task allocation plan. We introduce a recovery-latency aware task allocation problem (RTAP) that seeks task allocation plans for stream topologies that will achieve guaranteed recovery latencies. We discuss the difference between RTAP and classic task allocation problems and present a heuristic algorithm with a computational complexity of O(n log² n) to solve the problem. Extensive experiments were conducted to verify the correctness and effectiveness of our approach. It improves the resource usage by 15%–20% on average.