Hallucination-driven design of system-level laboratory projects for computer science education

The rapid development of large language models (LLMs) has brought unprecedented opportunities and challenges to higher education experimental teaching. While LLMs can provide personalized learning experiences, automated assessment, and immediate feedback, they also create substantial challenges, including students' over-reliance on AI tools, declining autonomous programming abilities, and increased academic misconduct. This study proposes an innovative hallucination self-testing framework that deliberately triggers LLM hallucinations under controlled conditions to enhance students' autonomous programming capabilities while ensuring educational fairness. The research addresses three critical challenges: designing experimental projects that can evaluate students' system coding capabilities while preventing excessive reliance on AI tools; balancing the convenience of LLM-assisted learning with academic integrity; and evaluating students' true abilities under LLM assistance. The objective is to develop a novel approach that integrates LLM hallucination characteristics into the design of database system kernel experimental projects, creating a framework that can effectively control the introduction of large model hallucinations into specified experimental projects to drive students' autonomous learning exploration.

The study focuses on database system kernel code development across three core domains: storage engines, query engines, and transaction processing. The research develops a multidimensional hallucination assessment framework with three core metrics: factual inconsistency, logical contradiction, and semantic absurdity, each quantified through independent mathematical models. The LLM hallucination self-testing framework is designed with the LLM alternately playing the role of student, question designer, and answer provider. The framework operates through iterative optimization, where initial experimental questions undergo multiple rounds of testing and refinement, with the question designer analyzing hallucination patterns from student solutions and optimizing question formulations to maximize hallucination triggers while maintaining educational value. The experimental validation involves designing specific tasks across the three core database system domains, implementing the self-testing framework, and measuring the effectiveness through compilation error rates, logical error rates, autonomous programming ratios, and knowledge coverage metrics.

The experimental validation demonstrates obvious improvements in both hallucination triggering effectiveness and educational outcomes. The optimized questions successfully increased LLM hallucination probabilities across all tested domains. Storage engine experiments showed compilation error rates rising from 35% to 68% and logical error rates increasing from 40% to 72%. Query engine experiments exhibited compilation error rates that climbed from 25% to 76%, with logical error rates rising from 18% to 49%. Transaction engine experiments demonstrated compilation error rates that increased from 42% to 80% alongside the logical error rate growth from 19% to 39%. More importantly, the research achieved its primary educational objectives: students' autonomous programming ratios increased substantially from the initial 40% to over 75% across all domains, while maintaining comprehensive knowledge coverage that improved from 74% to 96% due to increased student debugging processes. The hallucination self-testing framework successfully increased hallucination trigger probabilities by 25%–40% across different database system domains: storage engine hallucination probability increased from 35% to 60%, query engine hallucination probability increased from 30% to 52%, and transaction engine hallucination probability increased from 28% to 48%. The proposed method effectively reduces student dependency on LLMs while enhancing their autonomous programming capabilities and deepening their understanding of database kernel principles. The educational value of experimental projects was not compromised; rather, it was strengthened as students gained more comprehensive practical experience through autonomous problem-solving processes.

The proposed hallucination self-testing framework successfully achieves the intended goal of enhancing rare attack detection capabilities and improving system detection efficiency. The method addresses the fundamental challenge of balancing AI tool utility with educational integrity by creating a controlled environment where LLM limitations can be leveraged to promote student autonomy. The theoretical contributions include the formal mathematical representation of hallucination phenomena, the development of multidimensional assessment frameworks, and the establishment of quantitative evaluation metrics for AI-generated content quality. The practical contributions encompass the design principles for hallucination-driven question formulation, the construction of multirole collaborative self-testing processes, and empirical validation of the method's effectiveness in preventing academic dependence while enhancing practical capabilities. The framework establishes a foundation for future research in LLM–human collaboration, automated hallucination quantification, and cross-domain hallucination trigger mechanisms, with potential applications extending to other computer system courses such as operating systems and compiler principles. The findings suggest that traditional evaluation systems need innovation to adapt to LLM-assisted learning paradigms. This research provides educators with a sophisticated approach to leveraging LLM hallucinations for educational benefit through carefully designed experimental questions.