Exemplar-based image translation, which aims to transfer the style of an exemplar image to an input semantic image, is challenging and important in many applications. Most current methods build coarse correspondences and overlook extracting faithful style information from the exemplar image, leading to unsatisfactory results with style inconsistent with the exemplar image. In this paper, we propose a novel and efficient style mixture block to extract faithful style information and build reliable correspondences progressively. Specifically, instead of modeling explicit correspondences, we extract faithful style descriptors by considering global information about the exemplar features. Then, we generate coefficients for these style descriptors by modeling the interaction between the exemplar image and the input image, and efficiently compose these descriptors using the coefficients. The efficiency of the style mixture block allows a multi-scale architecture to extract and transform style descriptors at different resolutions, deforming the features of the exemplar image and refining the correspondences progressively. Experimental results on several datasets show that our SMixNet outperforms the current state-of-the-art, and is faster. Code is available for research purposes at https://github.com/Zhangjinso/SMixNet.

Novel space–time view synthesis for monocular video is a highly challenging task: both static and dynamic objects usually appear in the video, but only a single view of the current scene is available, resulting in inaccurate synthesis results. To address this challenge, we propose FRNeRF, a novel space–time view synthesis method with a fusion regularization field. Specifically, we design a 2D–3D fusion regularization field for the original dynamic neural field, which helps reduce blurring of dynamic objects in the scene. In addition, we add image prior features to the hierarchical sampling to solve the problem that the traditional hierarchical sampling strategy cannot obtain sufficient sampling points during training. We evaluate our method extensively on multiple datasets and show the results of dynamic space–time view synthesis. Our method achieves state-of-the-art performance both qualitatively and quantitatively. Code is available for research purposes at https://cic.tju.edu.cn/faculty/likun/projects/FRNerf.

Novel viewpoint image synthesis is very challenging, especially from sparse views, due to large changes in viewpoint and occlusion. Existing image-based methods fail to generate reasonable results for invisible regions, while geometry-based methods have difficulties in synthesizing detailed textures. In this paper, we propose STATE, an end-to-end deep neural network, for sparse view synthesis by learning structure and texture representations. Structure is encoded as a hybrid feature field to predict reasonable structures for invisible regions while maintaining original structures for visible regions, and texture is encoded as a deformed feature map to preserve detailed textures. We propose a hierarchical fusion scheme with intra-branch and inter-branch aggregation, in which spatio-view attention allows multi-view fusion at the feature level to adaptively select important information by regressing pixel-wise or voxel-wise confidence maps. By decoding the aggregated features, STATE is able to generate realistic images with reasonable structures and detailed textures. Experimental results demonstrate that our method achieves qualitatively and quantitatively better results than state-of-the-art methods. Our method also enables texture and structure editing applications benefiting from implicit disentanglement of structure and texture. Our code is available at http://cic.tju.edu.cn/faculty/likun/projects/STATE.