This paper investigates the optimal reinsurance and investment problem with delay in the framework of the non-zero-sum stochastic differential game, where the game is considered between two insurers characterized by similar ambiguity-averse preferences. Both insurers maximize their respective $\alpha$-maxmin mean-variance criterion in the market. The criterion is time-inconsistent and we derive the equilibrium reinsurance-investment strategies and value functions by the extended HJB equations. Finally, some numerical examples and sensitivity analysis are presented to demonstrate the effects of model parameters on the equilibrium strategy. We find the delay factor and the various attitudes of decision-makers toward ambiguity have a great influence on the final strategy. Moreover, the insurer's investment behavior will be more aggressive the more intense its competition with other insurers and the greater its ambiguity-seeking.