The lot-streaming flowshop scheduling problem with equal-size sublots (ELFSP) is a significant extension of the classic flowshop scheduling problem, focusing on optimize makespan. In response, an improved dynamic Q-learning (IDQL) algorithm is proposed, utilizing makespan as feedback. To prevent blind search, a dynamic $\epsilon$-greedy search strategy is introduced. Additionally, the Nawaz-Enscore-Ham (NEH) algorithm is employed to diversify solution sets, enhancing local optimality. Addressing the limitations of the dynamic $\epsilon$-greedy strategy, the Glover operator complements local search efforts. Simulation experiments, comparing the IDQL algorithm with other intelligent algorithms, validate its effectiveness. The performance of the IDQL algorithm surpasses that of its counterparts, as evidenced by the experimental analysis. Overall, the proposed approach offers a promising solution to the complex ELFSP, showcasing its capability to efficiently minimize makespan and optimize scheduling processes in flowshop environments with equal-size sublots.