Let ${\mathfrak{R}}_{l,k}={\mathbb{F}}_{p^m}[u_1,u_2,\cdots ,u_k]/⟨u_i^l=u_i,u_i{u}_j=u_j{u}_i=0⟩$, where $p$ is a prime, $l$ is a positive integer, $(l-1)\mid (p-1)$ and $1\le i,j\le k$. First, we define a Gray map ${\varphi }_{l,k}$ from ${\mathfrak{R}}_{l,k}^n$ to ${\mathbb{F}}_{p^m}^{((l-1)k+1)n}$, and study its Gray image. Further, we study the algebraic structure of $\sigma$-self-orthogonal and $\sigma$-dual-containing constacyclic codes over ${\mathfrak{R}}_{l,k}$, and give the necessary and sufficient conditions for $\lambda$-constacyclic codes over ${\mathfrak{R}}_{l,k}$ to satisfy $\sigma$-self-orthogonal and $\sigma$-dual-containing. Finally, we construct quantum codes from $\sigma$-dual-containing constacyclic codes over ${\mathfrak{R}}_{l,k}$ using the CSS construction or Hermitian construction and compare new codes our obtained better than the existing codes in some recent references.