---
title: Refining sentence selection for automatic cloze question generation with large language models 
description: "AI-generated questions in context: A contextualized investigation using platform data, student feedback, and faculty observations."
---

[Research Publications ](https://research.vitalsource.com/publications)

# [Refining sentence selection for automatic cloze question generation with large language models ](https://research.vitalsource.com/publications/refining-sentence-selection-for-automatic-cloze-question-generation-with-large-language-models)

 Written by [Rachel Van Campenhout](https://research.vitalsource.com/publications/author/rachel-van-campenhout) | Jul 17, 2025 5:00:00 AM

## Abstract

Formative practice embedded in textbooks has been shown to substantially enhance learning, yet manually authoring high-quality questions at scale is prohibitive. Recent advances in automatic question generation (AQG) have enabled large-scale production of formative practice questions. This study investigates whether a large language model (LLM) can improve selection of textbook sentences for fill-in-the-blank cloze questions by identifying those that lead to higher rates of negative student feedback. A larger LLM was employed to expedite prompt engineering for sentence classification by a smaller model. Over 1.3 million student-question sessions spanning 2,500 textbooks were analyzed using an explanatory logistic regression model. Questions derived from sentences the filter rejects were over three times as likely to receive a thumbs down rating by students, even after controlling for previously established causal factors. This finding indicates that an LLM-based filter can be integrated into existing rule-based AQG pipelines to remove flawed items and raise overall question quality in large-scale educational applications with minimal additional overhead. 

[View full post](https://research.vitalsource.com/publications/refining-sentence-selection-for-automatic-cloze-question-generation-with-large-language-models)

```json
{
  "@context" : "http://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "Rachel Van Campenhout"
  },
  "dateModified" : "2026-05-18T10:26:32.256Z",
  "datePublished" : "2025-07-17T05:00:00Z",
  "headline" : "Refining sentence selection for automatic cloze question generation with large language models ",
  "image" : {
    "@type" : "ImageObject",
    "height" : 60,
    "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
    "width" : 60
  },
  "mainEntityOfPage" : "https://research.vitalsource.com/publications/refining-sentence-selection-for-automatic-cloze-question-generation-with-large-language-models",
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "height" : 60,
      "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
      "width" : 60
    },
    "name" : "Research Publications"
  }
}
```