Integrating formative practice questions with textbook content at frequent intervals creates an active learning environment that is more effective for student learning. Advances in artificial intelligence have made it possible to develop automatic question generation systems robust enough for use with students at scale. In this paper, we analyze five types of automatically generated questions using data from hundreds of thousands of students across more than eight thousand textbooks. The difficulty and persistence performance metrics of these questions build on previous research and reveal insights into question performance and student behavior. Metacognitive tutorial activities are also generated, and investigation into students’ open ended responses show differences in how students apply what they have learned from the text.