Large-scale machine learning (ML) clusters deploy hundreds of models packaged as large container images, making scheduling and image pulling dominant contributors to deployment latency. Current systems adopt a hybrid-granularity strategy: they schedule using image-level metadata, reuse on-node layer caches, and lazily fetch content at the file or chunk level. This fragmentation leads to suboptimal cross-image reuse. We show that an intermediate unit, software granularity, can be uniformly applied at scheduling, caching, and pulling to improve deployment performance. We present Containerlet, a software-scoped layer that reorganizes standard images into one private layer plus multiple reusable per-software units that reside in compatible registries and are pulled by unmodified runtimes. Containerlet promotes cross-image reuse while bounding the number of units that the scheduler must consider. We design a file backtracking method to convert standard images into Containerlet images, which includes one private layer and multiple Containerlets. We also extend Kubernetes and container runtime to support software-granular scheduling, caching, and lazy pulling. End-to-end deployment experiments on diverse ML images and production cluster traces show consistent gains, reducing deployment overhead by 18% compared with state-of-the-art systems.

Edge machine learning creates a new computational paradigm by enabling the deployment of intelligent applications at the network edge. It enhances application efficiency and responsiveness by performing inference and training tasks closer to data sources. However, it encounters several challenges in practice. The variance in hardware specifications and performance across different devices presents a major issue for the training and inference tasks. Additionally, edge devices typically possess limited network bandwidth and computing resources compared with data centers. Moreover, existing distributed training architectures often fail to consider the constraints of resources and communication efficiency in edge environments. In this paper, we propose DSparse, a method for distributed training based on sparse update in edge clusters with various memory capacities. It aims at maximizing the utilization of memory resources across all devices within a cluster. To reduce memory consumption during the training process, we adopt sparse update to prioritize the updating of selected layers on the devices in the cluster, which not only lowers memory usage but also reduces the data volume of parameters and the time required for parameter aggregation. Furthermore, DSparse utilizes a parameter aggregation mechanism based on multi-process groups, subdividing the aggregation tasks into AllReduce and Broadcast types, thereby further reducing the communication frequency for parameter aggregation. Experimental results using the MobileNetV2 model on the CIFAR-10 dataset demonstrate that DSparse reduces memory consumption by an average of 59.6% across seven devices, with a 75.4% reduction in parameter aggregation time, while maintaining model precision.