We study the Maxwell system with localized conductivity $\sigma$ and the boundary conditions of a perfect conductor on a simply connected domain $\mathrm{\Omega }$, assuming that there are no electric charges off the support of $\sigma$. For matrix-valued permittivity $\epsilon$ and permeability $\mu$, we show strong stability of the underlying semigroup by checking the spectral criteria of the Arendt–Batty–Lyubich–Vũ Theorem. If $\epsilon =\mu =1$, $\mathrm{\Omega }$ is the cube $(0,\pi {)}^3$ and $\text{supp}\sigma$ contains a strip, the semigroup is polynomially stable of rate $\frac{1}{2}$. To derive this result, we establish the resolvent estimate of the Borichev–Tomilov Theorem using an orthonormal basis of eigenfunctions of the Maxwell operator for $\sigma =0$.