AI Chat Paper
Note: Please note that the following content is generated by AMiner AI. SciOpen does not take any responsibility related to this content.
{{lang === 'zh_CN' ? '文章概述' : 'Summary'}}
{{lang === 'en_US' ? '中' : 'Eng'}}
Chat more with AI
Powered by AMiner. Clicking this button will take you away from SciOpen.
Home Computational Visual Media Article
PDF (28.3 MB)
Cite
EndNote(RIS) BibTeX
Collect
Submit Manuscript AI Chat Paper
Show Outline
Outline
Show full outline
Hide outline
Outline
Show full outline
Hide outline
Research Article | Open Access

A simple and effective filtering scheme for improving neural fields

College of Computer and Data Science, Fuzhou University, Fuzhou 350108, China
Show Author Information

Graphical Abstract

Abstract

Neural fields, also known as coordinate-based multi-layer perceptrons (MLPs), have recently achieved impressive results in representing low-dimensional data. Unlike convolutional neural networks (CNNs), MLPs are globally connected and lack local control; adjusting a local region leads to global changes. Therefore, improving local neural fields usually leads to a dilemma: filtering out local artifacts can simultaneously smooth away desired details. Our solution is a new filtering technique that consists of two counteractive operators: a smoothing operator that provides global smoothing for better generalization and a recovery operator that provides better controllability for local adjustments. We found that using either operator alone could lead to an increase in noisy artifacts or oversmoothed regions. By combining the two operators, smoothing and sharpening can be adjusted to first smooth the entire region and then recover fine-grained details in the overly smoothed regions. Thus, our filter helps neural fields remove significant noise while enhancing the details. We demonstrate the benefits of our filter on various tasks, where it shows significant improvements over state-of-the-art methods. Moreover, our filter provides a better performance in terms of convergence speed and network stability.

Keywords

neural fieldsneural filterimplicit neural representationrepresentation learning

References

[1]

Xie, Y.; Takikawa, T.; Saito, S.; Litany, O.; Yan, S.; Khan, N.; Tombari, F.; Tompkin, J.; Sitzmann, V.; Sridhar, S. Neural fields in visual computing and beyond. Computer Graphics Forum Vol. 41, No. 2, 641−676, 2022.
Crossref Google Scholar
[2]
Basri, R.; Galun, M.; Geifman, A.; Jacobs, D.; Kasten, Y.; Kritchman, S. Frequency bias in neural networks for input of non-uniform density. In: Proceedings of the 37th International Conference on Machine Learning, 685−694, 2020.
[3]
Rahaman, N.; Baratin, A.; Arpit, D.; Draxler, F.; Lin, M.; Hamprecht, F.; Bengio, Y.; Courville, A. On the spectral bias of neural networks. In: Proceedings of the 36th International Conference on Machine Learning, 5301−5310, 2019.
[4]
Tancik, M.; Srinivasan, P. P.; Mildenhall, B.; Fridovich-Keil, S.; Raghavan, N.; Singhal, U.; Ramamoorthi, R.; Barron, J. T.; Ng, R. Fourier features let networks learn high frequency functions in low dimensional domains. arXiv preprint arXiv: 2006.10739, 2020.
[5]
Sitzmann, V.; Martel, J. N. P.; Bergman, A. W.; Lindell, D. B.; Wetzstein, G. Implicit neural representations with periodic activation functions. arXiv preprint arXiv: 2006.09661, 2020.
[6]
Hertz, A.; Perel, O.; Giryes, R.; Sorkine-Hornung, O.; Cohen-Or, D. SAPE: Spatially-adaptive progressive encoding for neural optimization. In: Proceedings of the 35th Conference on Neral Information Processing System, 8820−8832, 1999.
[7]
Mehta, I.; Gharbi, M.; Barnes, C.; Shechtman, E.; Ramamoorthi, R.; Chandraker, M. Modulated periodic activations for generalizable local functional representations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 14214−14223, 2021.
Crossref
[8]
Fathony, R.; Sahu, A. K.; Willmott, D.; Kolter, J. Z. Multiplicative filter networks. In: Proceedings of the International Conference on Learning Representations, 2020.
[9]
Lindell, D. B.; Van Veen, D.; Park, J. J.; Wetzstein, G. Bacon: Band-limited coordinate networks for multiscale scene representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 16252−16262, 2022.
Crossref
[10]
Chan, E. R.; Monteiro, M.; Kellnhofer, P.; Wu J.; Wetzstein, G. Pi-GAN: Periodic implicit generative adversarial networks for 3D-aware image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5799−5809, 2021.
Crossref
[11]

Mildenhall, B.; Srinivasan, P. P.; Tancik, M.; Barron, J. T.; Ramamoorthi, R.; Ng, R. NeRF: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM Vol. 65, No. 1, 99−106, 2021.
Crossref Google Scholar
[12]
Chen, Y.; Liu, S.; Wang, X. Learning continuous image representation with local implicit image function. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8628−8638, 2021.
Crossref
[13]
Reddy, P.; Zhang, Z.; Wang, Z.; Fisher, M.; Jin, H.; Mitra, N. A multi-implicit neural representation for fonts. In: Processing of the 35th Conference on Neural Information Processing Systems, 12637−12647, 2021.
[14]
Park, J. J.; Florence, P.; Straub, J.; Newcombe, R.; Lovegrove, S. DeepSDF: Learning continuous signed distance functions for shape representation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 165−174, 2019.
Crossref
[15]
Mescheder, L.; Oechsle, M.; Niemeyer, M.; Nowozin, S.; Geiger, A. Occupancy networks: Learning 3D reconstruction in function space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4460−4470, 2019.
Crossref
[16]
Chen, Z.; Zhang, H. Learning implicit fields for generative shape modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5939−5948, 2019.
Crossref
[17]
Atzmon, M.; Lipman, Y. SAL: Sign agnostic learning of shapes from raw data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2562−2571, 2020.
Crossref
[18]
Gropp, A.; Yariv, L.; Haim, N.; Atzmon, M.; Lipman, Y. Implicit geometric regularization for learning shapes. In: Proceedings of the 37th International Conference on Machine Learning, 3789−3799, 2020.
[19]
Sitzmann, V.; Zollhöefer, M.; Wetzstein, G. Scene representation networks: Continuous 3D-structure-aware neural scene representations. In: Proceedings of the 33rd Conference on Neural Information Processing Systems, 2022.
[20]
Peng, S.; Zhang, Y.; Xu, Y.; Wang, Q.; Shuai, Q.; Bao, H.; Zhou, X. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9054−9063, 2021.
Crossref
[21]
Martin-Brualla, R.; Radwan, N.; Sajjadi, M. S. M.; Barron, J. T.; Dosovitskiy, A.; Duckworth, D. NeRF in the wild: Neural radiance fields for unconstrained photo collections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7210−7219, 2021.
Crossref
[22]
Yu, A.; Ye, V.; Tancik, M.; Kanazawa, A. pixelNeRF: Neural radiance fields from one or few images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4578−4587, 2021.
Crossref
[23]

Bemana, M.; Myszkowski, K.; Seidel, H. P.; Ritschel, T. X-fields: Implicit neural view-, light- and time-image interpolation. ACM Transactions on Graphics Vol. 39, No, 6 Article No. 257, 2020.
Crossref Google Scholar
[24]
Williams, F.; Trager, M.; Bruna, J.; Zorin, D. Neural splines: Fitting 3D surfaces with infinitely-wide neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9949−9958, 2021.
Crossref
[25]
Williams, F.; Schneider, T.; Silva, C.; Zorin, D.; Bruna, J.; Panozzo, D. Deep geometric prior for surface reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10130−10139, 2019.
Crossref
[26]
Xu, Q.; Wang, W.; Ceylan, D.; Mech, R.; Neumann, U. DISN: Deep implicit surface network for high-quality single-view 3D reconstruction. In: Proceedings of the 33rd Conference on Neural Information Processing Systems, 490−500, 2019.
[27]
Xu, Y.; Fan, T.; Yuan, Y.; Singh, G. Ladybird: Quasi-Monte Carlo sampling for deep implicit field based 3D reconstruction with symmetry. In: Computer Vision − ECCV 2020. Lecture Notes in Computer Science, Vol. 12346. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J.-M. Eds. Springer Cham, 248−263, 2020.
Crossref
[28]
Saito, S.; Huang Z.; Natsume, R.; Morishima, S.; Li H.; Kanazawa, A. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2304−2314, 2019.
Crossref
[29]
Chen, Y.; Fernando, B.; Bilen, H.; Mensink, T.; Gavves, E. Neural feature matching in implicit 3D representations. In: Proceedings of the 38th International Conference on Machine Learning, 1582−1593, 2021.
[30]
Zhi, S.; Laidlow, T.; Leutenegger, S.; Davison, A. J. In-place scene labelling and understanding with implicit scene representation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, 15838−15847, 2021.
Crossref
[31]
Wang, P.; Liu, Y.; Yang, Y.; Tong, X. Spline positional encoding for learning 3D implicit signed distance fields. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence, 1091−1097, 2021.
Crossref
[32]
Chabra, R.; Lenssen, J. E.; Ilg, E.; Schmidt, T.; Straub, J.; Lovegrove, S.; Newcombe, R. A. Deep local shapes: Learning local SDF priors for detailed 3D reconstruction. In: Computer Vision − ECCV 2020. Lecture Notes in Computer Science, Vol. 12374. Vedaldi, A.; Bischof, H.; Brox, T.; Frahm, J.-M. Eds. Springer Cham, 608−625, 2020.
Crossref
[33]
Genova, K.; Cole, F.; Sud, A.; Sarna, A.; Funkhouser, T. Local deep implicit functions for 3D shape. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4856−4865, 2020.
Crossref
[34]
Jiang C.; Sud, A.; Makadia, A.; Huang J.; Nießner, M.; Funkhouser, T. Local implicit grid representations for 3D scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 6000−6009, 2020.
Crossref
[35]
Jampani, V.; Kiefel, M.; Gehler, P. V. Learning sparse high dimensional filters: Image filtering, dense CRFs and bilateral neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4452−4461, 2016.
Crossref
[36]
Zhang, J.; Pan, J.; Lai, W. S.; Lau, R. W. H.; Yang, M. H. Learning fully convolutional networks for iterative non-blind deconvolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 6969−6977, 2017.
Crossref
[37]
Yoon, Y.; Jeon, H. G.; Yoo, D.; Lee, J. Y.; Kweon, I. S. Learning a deep convolutional network for light-field image super-resolution. In: Proceedings of the IEEE International Conference on Computer Vision Workshop, 57−65, 2015.
Crossref
[38]

Gharbi, M.; Chen J.; Barron, J. T.; Hasinoff, S. W.; Durand, F. Deep bilateral learning for real-time image enhancement. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 118, 2017.
Crossref Google Scholar
[39]
Chen, Q.; Xu, J.; Koltun, V. Fast image processing with fully-convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, 2516−2525, 2017.
Crossref
[40]

Li, Y.; Huang, J. B.; Ahuja, N.; Yang, M. H. Joint image filtering with deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 41, No. 8, 1909−1923, 2019.
Crossref Google Scholar
[41]

Andersson, P.; Nilsson, J.; Akenine-Möller, T.; Oskarsson, M.; Åström, K.; Fairchild, M. D. FLIP: A difference evaluator for alternating images. Proceedings of the ACM on Computer Graphics and Interactive Techniques Vol. 3, No. 2, Article No. 15, 2020.
Crossref Google Scholar
[42]

Lorensen, W. E.; Cline, H. E. Marching cubes: A high resolution 3D surface construction algorithm. ACM SIGGRAPH Computer Graphics Vol. 21, No. 4, 163−169, 1987.
Crossref Google Scholar
[43]
Koch, S.; Matveev, A.; Jiang, Z.; Williams, F.; Artemov, A.; Burnaev, E.; Alexa, M.; Zorin, D.; Panozzo, D. ABC: A big CAD model dataset for geometric deep learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9601−9611, 2019.
Crossref
[44]
Schönberger, J. L.; Frahm, J. M. Structure-from-motion revisited. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4104−4113, 2016.
Crossref
Computational Visual Media
Volume 11 Issue 2,
April 2025
Pages 343-359
DOI: 10.26599/CVM.2025.9450376
Cite this article:
Zhuang Y. A simple and effective filtering scheme for improving neural fields. Computational Visual Media, 2025, 11(2): 343-359. https://doi.org/10.26599/CVM.2025.9450376

28

Views

2

Downloads

0

Crossref

0

Web of Science

0

Scopus

0

CSCD

Altmetrics
Received: 28 February 2023
Accepted: 30 August 2023
Published: 08 May 2025
© The Author(s) 2025.

This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

To submit a manuscript, please go to https://jcvm.org.
Return