Abstract
There is a dire need for replication research in the learning sciences, as methods put forth for increasing student learning should be unequivocally grounded in reproducible, reliable research. Learning science research is not only a critical input in the learning engineering process during the development of educational technology tools, such as courseware, but also as an output after student data have been analyzed to determine if the learning methods used were effective for students in their natural learning context. Furthermore, research that can provide causal evidence that a method of learning is effective for students should be reproduced—and the generality for its use expanded—so that methods that cause learning gains can be widely applied. One such method is the doer effect: the principle that students who engage with more practice have higher learning gains than those who only read expository text or watch video. This effect has been shown to be causal in prior research through statistical modeling using data mined from natural learning contexts. The goal of this paper is to replicate this research using a large-scale data set from courseware used at a major online university. The learning-by-doing data recorded by the courseware platform were combined with final exam data to replicate the statistical model of the causal doer effect study. Results from this analysis similarly point to a causal relationship between doing practice and learning outcomes. The implications of these doer effect results and future learning science research using large-scale data analytics will be discussed.
