Considering the demand of distributed satellite clusters for high-speed information communication in the future, this paper establishes a laser network model based on optical multibeam antenna. At present, there are still some networking and reconstruction problems, such as network connectivity, duration, and stability. To address them, the paper develops a multiobjective optimization model for the laser networking of distributed satellite clusters, which aims to maximize network connectivity and network duration and minimize the perturbation of the network connection matrix. The model is constructed under the constraints of multibeam antenna capability, the visibility of satellites in clusters, and network connectivity. From the perspectives of the optimization effect and timeliness of the optimization algorithm, a deep reinforcement learning algorithm is proposed, which is based on a double-layer Markov decision model, to meet the needs of on-orbit intelligent networking and dynamic reconstruction of distributed satellite clusters. Simulation results show that the algorithm features flexible architecture, excellent networking performance, and strong real-time performance. When the optimization results are similar, the proposed algorithm outperforms the nonsorted genetic algorithm Ⅱ algorithm and the particle swarm optimization algorithm in terms of solution speed.