--- title: "Open-ended questions need personalized feedback: Analyzing LLM-enabled features with student data" description: "AI-generated questions in context: A contextualized investigation using platform data, student feedback, and faculty observations." --- < Back to Publications # Open-ended questions need personalized feedback: Analyzing LLM-enabled features with student data ## Abstract Large language models (LLMs) offer new opportunities to support deeper learning through open-ended, formative practice. This paper investigates two novel types of automatically generated questions: compare-and-contrast prompts and student-authored exam questions. These question types are integrated into an ereader platform alongside conventional fill-in-the-blank items. To enable meaningful interaction with these open-ended tasks, an LLM is used to generate personalized feedback grounded in textbook content. A dataset of more than 90,000 student-question interactions is analyzed to evaluate how these new question types perform in terms of engagement, difficulty, persistence, and non-genuine responses, and how students interact with the LLM-generated feedback. Results are compared across contexts where questions were assigned as part of a course versus used voluntarily. Assigned usage dramatically increases engagement and improves performance across most metrics. To understand how students respond to the feedback itself, timing and textual overlap between the initial LLM-generated feedback and the student’s second attempt are examined, revealing distinct patterns of reflection, revision, and potential feedback reuse. These results highlight both the promise and complexity of using LLMs to expand the cognitive scope of automated formative practice while maintaining pedagogical value at scale.' [Read Publication](https://ceur-ws.org/Vol-4006/paper5.pdf) [View Data Set](https://github.com/vitalsource/data/tree/0afe93ad37901195415b288e5b96e9221221d90d/aied-2025-evallac) ##### Date July 26, 2025 ##### Authors - Rachel Van Campenhout - Jeffrey S. Dittel - Bill Jerome - Michelle W. Clark - Benny G. Johnson ##### Research Areas [Automatic Question Generation](https://research.vitalsource.com/publications/tag/automatic-question-generation) [Generative AI](https://research.vitalsource.com/publications/tag/generative-ai) [Feedback](https://research.vitalsource.com/publications/tag/feedback) ##### Venue Second Workshop on Automated Evaluation of Learning and Assessment Content at the 26th International Conference on Artificial Intelligence in Education ##### Citation Van Campenhout, R., Dittel, J. S., Jerome, B., Clark, M. W., & Johnson, B. G. (2025). Open-ended questions need personalized feedback: Analyzing LLM-enabled features with student data. *Proceedings of the Second Workshop on Automated Evaluation of Learning and Assessment Content (EvalLAC '25), Vol. 4006*. CEUR-WS. [https://ceur-ws.org/Vol-4006/paper5.pdf](https://ceur-ws.org/Vol-4006/paper5.pdf) ##### Share [![](https://research.vitalsource.com/hubfs/2021_Research/Social%20-%20Facebook.svg)](https://www.facebook.com/sharer/sharer.php?u=https://research.vitalsource.com/publications/open-ended-questions-need-personalized-feedback-analyzing-llm-enabled-features-with-student-data) [![](https://research.vitalsource.com/hubfs/2021_Research/Social%20-%20LinkedIn.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https://research.vitalsource.com/publications/open-ended-questions-need-personalized-feedback-analyzing-llm-enabled-features-with-student-data) [![](https://research.vitalsource.com/hubfs/2021_Research/Social%20-%20Twitter.svg)](https://twitter.com/intent/tweet?url=https://research.vitalsource.com/publications/open-ended-questions-need-personalized-feedback-analyzing-llm-enabled-features-with-student-data&text=) ### Related Publications #### Search our catalog of recent publications authored by our team. **2026** ##### [Extending an automatic question generation pipeline with LLM-based free-response tasks: An analysis of performance metrics using student data](https://research.vitalsource.com/publications/extending-an-automatic-question-generation-pipeline-with-llm-based-free-response-tasks-an-analysis-of-performance-metrics-using-student-data) **CSEDU 2026 – CSEDU 18th International Conference on Computer Supported Education** ![profile](https://research.vitalsource.com/hubfs/2021_Research/Home/profile.svg) Rachel Van Campenhout, Jeffrey S. Dittel, Bill Jerome, Benny G. Johnson (2026) [Automatic Question Generation](https://research.vitalsource.com/publications/tag/automatic-question-generation) [Generative AI](https://research.vitalsource.com/publications/tag/generative-ai) [Teaching & Learning](https://research.vitalsource.com/publications/tag/teaching-learning) **2026** ##### [Student attitudes and perceptions of AI and AI-generated practice in the classroom](https://research.vitalsource.com/publications/student-attitudes-and-perceptions-of-ai-and-ai-generated-practice-in-the-classroom) **The IAFOR International Conference on Education – Hawaii 2026** ![profile](https://research.vitalsource.com/hubfs/2021_Research/Home/profile.svg) Rachel Van Campenhout, Michelle Clark (2026) [Automatic Question Generation](https://research.vitalsource.com/publications/tag/automatic-question-generation) [Generative AI](https://research.vitalsource.com/publications/tag/generative-ai) [Teaching & Learning](https://research.vitalsource.com/publications/tag/teaching-learning) **2025** ##### [AI-generated formative practice and feedback: Performance benchmarks and applications in higher education](https://research.vitalsource.com/publications/ai-generated-formative-practice-and-feedback-performance-benchmarks-and-applications-in-higher-education) **Artificial Intelligence in Measurement and Education Conference (AIME-CON) ** ![profile](https://research.vitalsource.com/hubfs/2021_Research/Home/profile.svg) Rachel Van Campenhout, Michelle Clark, Jeffrey S. Dittel, Bill Jerome, Nick Brown, Benny G. Johnson (2025) [Automatic Question Generation](https://research.vitalsource.com/publications/tag/automatic-question-generation) [Generative AI](https://research.vitalsource.com/publications/tag/generative-ai) [Feedback](https://research.vitalsource.com/publications/tag/feedback) **2025** ##### [Improving automatically generated fill-in-the-blank answer selection with an LLM-based agreement filter](https://research.vitalsource.com/publications/improving-automatically-generated-fill-in-the-blank-answer-selection-with-an-llm-based-agreement-filter) **Educational Data Mining 2025 Workshop on Causal Inference** ![profile](https://research.vitalsource.com/hubfs/2021_Research/Home/profile.svg) Benny G. Johnson, Jeffrey S. Dittel, Rachel Van Campenhout (2025) [Automatic Question Generation](https://research.vitalsource.com/publications/tag/automatic-question-generation) [Generative AI](https://research.vitalsource.com/publications/tag/generative-ai) **2025** ##### [Scaling effective characteristics of ITSs: A preliminary analysis of LLM-based personalized feedback](https://research.vitalsource.com/publications/scaling-effective-characteristics-of-itss-a-preliminary-analysis-of-llm-based-personalized-feedback) **International Conference on Intelligent Tutoring Systems** ![profile](https://research.vitalsource.com/hubfs/2021_Research/Home/profile.svg) Rachel Van Campenhout, Jeffrey S. Dittel, Benny G. Johnson (2025) [Automatic Question Generation](https://research.vitalsource.com/publications/tag/automatic-question-generation) [Generative AI](https://research.vitalsource.com/publications/tag/generative-ai) [Feedback](https://research.vitalsource.com/publications/tag/feedback) **2025** ##### [Open-ended questions need personalized feedback: Analyzing LLM-enabled features with student data](https://research.vitalsource.com/publications/open-ended-questions-need-personalized-feedback-analyzing-llm-enabled-features-with-student-data) **Second Workshop on Automated Evaluation of Learning and Assessment Content at the 26th International Conference on Artificial Intelligence in Education** ![profile](https://research.vitalsource.com/hubfs/2021_Research/Home/profile.svg) Rachel Van Campenhout, Jeffrey S. Dittel, Bill Jerome, Michelle W. Clark, Benny G. Johnson (2025) [Automatic Question Generation](https://research.vitalsource.com/publications/tag/automatic-question-generation) [Generative AI](https://research.vitalsource.com/publications/tag/generative-ai) [Feedback](https://research.vitalsource.com/publications/tag/feedback) [See all publications](https://research.vitalsource.com/publications)