Automatic test paper generation method based on cognitive diagnosis and LLM-optimized genetic algorithm

To enhance the quality of test paper generation by teachers and the efficiency of teaching as well as optimize teaching resources, numerous scholars at home and abroad have conducted in-depth research on automatic test paper generation techniques. Previously proposed algorithms for automatic test paper generation include random selection, backtracking-based automatic test paper generation, and optimization algorithm–based automatic test paper generation. These traditional algorithms did not consider the overall cognitive level of classes, resulting in low efficiency in test paper generation and uneven quality of test papers, which affected the accurate assessment of students’ learning outcomes. To address these issues, this paper proposes an automatic test paper generation method based on cognitive diagnosis and a large language model (LLM)-optimized genetic algorithm.

First, leveraging the powerful data processing and analysis capabilities of pretrained LLMs, this paper proposes the use of fine-tuning techniques and question-answering mechanisms of LLMs for automatic test paper generation. This study explores the impact of different prompts, model parameter sizes, and fine-tuning techniques on automatic test paper generation tasks, ultimately selecting the Qwen LLM with 7 billion parameters and using the QLoRA fine-tuning technique for automatic test paper generation. This method can generate test papers efficiently, but the quality of the test papers is somewhat inferior to that of traditional genetic algorithms. Therefore, an improved automatic test paper generation method is proposed based on cognitive diagnosis and an LLM-optimized genetic algorithm. The test papers generated via the Qwen LLM are used as the initial population, and a fitness calculation method for test paper generation is developed. The genetic operators are designed with the idea of “elitism”, and the design of the selection, crossover, and mutation operators is improved to always retain the best individuals in the population.

Experimental results show that the traditional genetic algorithm takes 34.586 s to generate a test paper with a success rate of 83%, while the LLM-based automatic test paper generation algorithm completes the task in 8.565 s but with a slight decrease in the success rate to 81%. In comparison, the proposed genetic algorithm–based automatic test paper generation method combining cognitive diagnosis and LLM optimization not only significantly improves the efficiency of test paper generation to just 0.673 s but also achieves a success rate of 100%, fully meeting the needs of practical applications. This method significantly outperforms the aforementioned two algorithms, demonstrating dual advantages in efficiency and success rate.

By combining emerging LLM technology, cognitive diagnosis methods, and genetic algorithms for the requirements of automatic test paper generation, this method improves the efficiency of test paper generation while ensuring its quality and success rate. This innovative combination effectively circumvents the limitations of traditional test paper generation methods, providing strong technical support for teachers. It not only marks a significant advancement in information technology in the field of test paper generation but also adds robust momentum to educational reform, leading us toward a more efficient and intelligent educational future.