Quadrics are important in finite geometry and can be used to construct binary codes. In this paper, we first define an incidence matrix $M$ based on points and non-degenerate quadrics in the classical projective space PG $(n-1,q)$, where $q$ is a prime power. As a consequence, we establish a binary code $C(M)$ with the generator matrix $M$ and determine the dimension of $C(M)$ when $q$ and $n$ are both odd. In particular, we study the minimum distances of $C(M)$ and $C^{\perp }(M)$ in PG $(2,q)$ and give their upper bounds.