First, we construct a new type of feedforward neural network operators on finite intervals, and give the pointwise and global estimates of approximation by the new operators. The new operator can approximate the continuous functions with a very good rate, which can not be obtained by polynomial approximation. Second, we construct a new type of feedforward neural network operator on infinite intervals and estimate the rate of approximation by the new operators. Finally, we investigate the weighted approximation properties of the new operators on infinite intervals and show that our new neural networks are dense in a very wide class of functional spaces. Thus, we demonstrate that approximation by feedforward neural networks has some better properties than approximation by polynomials on infinite intervals.