$\mathcal{Z}-{\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{R}}^{+}$: A zero-shot style transfer method adjusting style distribution

Style transfer presents a significant challenge, primarily centered on identifying an appropriate style representation. Conventional methods employ style loss, derived from second-order statistics or contrastive learning, to constrain style representation in the stylized result. However, these pre-defined style representations often limit stylistic expression, leading to artifacts. In contrast to existing approaches, we have discovered that latent features in basic diffusion models inherently contain natural style and content distributions. This allows for direct extraction of style information and seamless integration of generative priors into the content image without the need for retraining. Our method adopts dual denoising paths to represent content and style references in the latent space, subsequently guiding the content image denoising process with style latent codes. We introduce a cross-attention reweighting module that utilizes local content features to query style image information best suited to the input patch, thereby aligning the style distribution of the stylized results with that of the style image. Furthermore, we have designed a scaled adaptive instance normalization approach to mitigate global inconsistencies in color distribution between style and stylized images. Through theoretical analysis and extensive experimentation, we demonstrate the effectiveness and superiority of $\mathcal{Z}-{\mathrm{S}\mathrm{T}\mathrm{A}\mathrm{R}}^{+}$, our diffusion-based zero-shot style transfer method using style distribution adjustment.