Motivated by the stochastic optimal control problem, in this paper we systematically study a class of mean-field backward stochastic Volterra integral equations with jumps (MF-BSVIEs with jumps, for short). More precisely, by a special kind of BSVIEs with jumps, we present the well-posedness of MF-BSVIEs with jumps in the sense of adapted M-solutions. For completeness, both the well-posedness of mean-field forward stochastic Volterra integral equations with jumps (MF-FSVIEs with jumps, for short) and the comparison theorem for MF-BSVIEs with jumps are directly proved. Based on this, by virtue of the duality principle, we are able to establish Pontryagin's type maximum principle for optimal control problems of MF-FSVIEs with jumps. We illustrate our results with an application to the linear-quadratic optimal control problem of MF-FSVIEs with jumps.