Self-orthogonal (SO) codes, including Hermitian self-orthogonal (HSO) codes, form an important class of linear codes, and such codes have close connections to other mathematical structures such as block designs, lattices, and sphere packings, which can also be used to construct quantum codes. Many scholars try to solve the problem of determining low-dimensional optimal HSO codes over small fields as it is done for optimal linear codes. Let $d_o(n,k)$ be the minimum distance of an optimal quaternary $[n,k]$ linear code, and $d_{so}(n,k)$ be that of an optimal quaternary $[n,k]$ HSO code. In this paper, we try to determine $d_{so}(n,5)$ for $n\ge 492$ by constructing quaternary $[n,5]$ HSO codes in detail. Some disjoint HSO blocks have been found from generator matrices of some special optimal HSO codes. These special optimal HSO codes are constructed from quaternary simplex codes and McDonald codes. Then, $[n,5]$ HSO codes have been constructed for $n\ge 492$, by removing those special blocks from the known optimal HSO codes. As a result, we could show $[n,5,d_{so}(n,5)]$ $=[n,5,2\lfloor \frac{d_o(n,5)}{2}\rfloor ]$ for $n\ge 492$.