Some basic theory on the duality of codes over two non-unital rings of order $6$, namely $H_{23}$ and $H_{32}$ is presented. For a code $\mathcal{C}$ over these rings, there is an associated binary code ${\mathcal{C}}_a$ and a ternary code ${\mathcal{C}}_b$. Self-orthogonal, self-dual, and quasi self-dual (QSD) codes over these rings are characterized using the associated codes ${\mathcal{C}}_a$ and ${\mathcal{C}}_b$, and a classification of self-orthogonal codes for short lengths is given. In addition, a building-up construction for self-orthogonal codes is presented, and cyclic and linear complementary dual (LCD) codes over the said rings are introduced.