Neural fields, also known as coordinate-based multi-layer perceptrons (MLPs), have recently achieved impressive results in representing low-dimensional data. Unlike convolutional neural networks (CNNs), MLPs are globally connected and lack local control; adjusting a local region leads to global changes. Therefore, improving local neural fields usually leads to a dilemma: filtering out local artifacts can simultaneously smooth away desired details. Our solution is a new filtering technique that consists of two counteractive operators: a smoothing operator that provides global smoothing for better generalization and a recovery operator that provides better controllability for local adjustments. We found that using either operator alone could lead to an increase in noisy artifacts or oversmoothed regions. By combining the two operators, smoothing and sharpening can be adjusted to first smooth the entire region and then recover fine-grained details in the overly smoothed regions. Thus, our filter helps neural fields remove significant noise while enhancing the details. We demonstrate the benefits of our filter on various tasks, where it shows significant improvements over state-of-the-art methods. Moreover, our filter provides a better performance in terms of convergence speed and network stability.