Abstract
The advent of robust, open generative AI systems has changed teaching, learning, and educational technology permanently and in ways that will continue to evolve in the coming years. AI has made it possible to dramatically increase the availability of learning science-based approaches-such as formative practice-to millions of students through educational technology platforms. When applied responsibly, generative AI can contribute to these advances in powerful ways. In this paper, we discuss the ways generative AI was used to support an existing rule-based AI automatic question generation system, both via theoretical framework and technical methods. New question types with immediate, personalized feedback were developed to harness the benefits of generative AI for error-specific feedback. Question data from 316 students across three courses were analyzed, examining metrics such as engagement, difficulty, persistence, and non-genuine responses. These analyses extend performance benchmarks for AI formative practice and suggest iterative improvement steps and avenues for future research.
