Mars exploration has increased the interest in bio-inspired flapping-wing micro aerial vehicles due to their high maneuverability and efficient performance in low Reynolds number environments. This work presents an optimization of the compound performance of two tandem pitching and heaving aerofoils in the Martian environment by using the hybrid method of the reinforcement learning (RL) and the immersed boundary-finite difference method (IB-FDM). The searching parameters include the horizontal and vertical spacing, phase shift, and mean angle of attack (AoA) of the hindwing at a Reynolds number of 1100. It is found that the optimal phase shifts lie close to in-phase and out-of-phase motions, while intermediate phase shifts tend to produce suboptimal results. For the in-phase cases, an increase in AoA and a decrease in horizontal spacing led to an increase in the compound performance. For the out-of-phase cases, a similar trend is observed until the optimal value is reached, after which the performance begins to decline. The effects of vertical spacing vary vastly on a case-to-case basis, depending on other motion and positional parameters.