The degradation of vanadium-based alloys during hydrogen sorption cycles is closely linked to defect accumulation (e.g., dislocation and lattice strain), yet the atomic-scale origins of such defects remain poorly understood. In present study, we reveal the crucial role of initial lattice distortion, quantified by the atomic size difference (δ), in the defect formation and accumulation of V-based alloys. Alloys with higher δ values exhibit accelerated attenuation of reversible hydrogen capacity (13.22% for δ = 4.32% vs. 5.60% for δ = 3.85% over 100 cycles), accompanied by increased plateau slope factors (S_f) and defect concentrations. High-resolution microscopy uncovers a two-stage defect evolution, associated with the generation of two types of nano-scale hierarchical structures. During the first cycle, nanograins with different spatial orientations show up, which geometrically leads to the formation of dislocations between the misoriented interfaces. In subsequent cycles, alternating nano-layered structures (1–2 nm thickness) gradually appear within the nanograins, resulting in the formation of subgrain boundaries accompanied with the local distortion and strains. These hierarchical nanostructures, driven by δ-dependent lattice distortion, are identified as the primary cause of the defects in alloys. This work provides a microstructure-guided strategy for designing durable hydrogen storage alloys by minimizing atomic size mismatch.