All solid-state lithium-sulfur batteries (ASSLSBs) are considered to be one of the most promising next-generation energy storage systems, due to the promises of high energy density and safety. Although the use of solid-state electrolytes could effectively suppress the "shuttle effect" and self-discharge of the conventional liquid lithium-sulfur (Li-S) battery, the commercialization of ASSLSBs has been seriously hampered by the electrolyte degradation, electrode/electrolyte interfacial deterioration, electrochemo-mechanical failure, lithium dendrite growth and electrode pulverization, etc. This paper provides a comprehensive review of recent research progresses on the solid-state electrolytes, sulfur-containing composite cathodes, lithium metal and lithium alloy anodes, and electrode/electrolyte interfaces in ASSLSBs. Specifically, lithium sulfide and metal sulfide as new active cathode materials, and lithium alloy as new anode materials are overviewed and analyzed. In addition, some newly developed interfacial modification strategies for addressing the electrode/electrolyte interfacial challenges are also outlined. Furthermore, an outlook on the future research and development of high-performance ASSLSBs are also presented.