Adaptive Diagnostics of Programming Misconceptions : Design and Construction of an Adaptive Assessment in Programming Education
Despite the growing relevance of programming education in recent years, there is still a lack of suitable, standardized assessments in the field. This shortcoming is especially present in the area of formative testing of novice programmers, where the focus is primarily on helping learners rather than on achieving performance-oriented outcomes. However, literature has shown that (many) learners encounter learning difficulties in the form of misconceptions during their learning process. Therefore, reliably and validly detecting them is a relevant task of assessment research. The detection would allow educators to plan further interventions to assist learners in overcoming their learning difficulties. My dissertation integrates findings from seven studies outlining a potential process for developing an adaptive assessment that employs item generation in the context of tracing elements of control flow. First, a static Rasch-scaled assessment is developed and evaluated for its effectiveness in detecting known misconceptions. Next, the selected item format is analyzed to understand its characteristics and the factors influencing item difficulty. Finally, the work describes and discusses the transition from the developed static Rasch-scaled assessment to an adaptive assessment that utilizes item generation, including an evaluation of the adaptive item selection process and the underlying item generator. The findings reveal that the selected item format can be used to detect misconceptions based on common patterns observed during the execution. A created test system can automatically analyze these patterns and provide programming educators with valuable insights into the cognitive processes of their learners. The developed adaptive assessment can decrease the time required to conduct a measurement by up to 49%. Moreover, switching from a static to an adaptive paradigm does not result in a significant loss of informativeness of the assessment.
Preview
Rights
Use and reproduction:
Please note that individual components of the publication may be subject to other licensing or copyright conditions.