MOPCGRL: Multi-Objective Procedural Content Generation via Reinforcement Learning

Online content generation enables automatic and adaptive creation of diverse and playable game content for maximizing player experience or testing Artificial Intelligence (AI) algorithms. Multiple diversity metrics have been formulated on different content facets in the literature, while some of them conflict with one another. Existing work addresses this multi-dimensional diversity nature by converting those metrics into one term that is further used to direct the training of content generators. However, each generator is trained to meet the preference specified by the weights and fails to fully interpret the relationships among these metrics or provide different trade-offs. This paper proposes a multi-objective procedural content generation via reinforcement learning to train a set of generators that create diverse game content in an online manner while balancing the trade-off between multiple diversity metrics with playability as a constraint. Our framework is compared with state-of-the-art approaches on the commonly used Mario-AI benchmark. Results show that our framework is capable of increasing the diversity of the generator distribution while accelerating the convergence during the early stages of model training. Our approach enables researchers, designers, and practitioners to gain a better understanding of the relationship among conflicting diversity metrics, allowing them to generate content more efficiently and accurately tailored to specific needs.