Exploring Diagnostic Capacity of a High Stakes Reading Comprehension Test: A Pedagogical Demonstration

Traditional analysis of test results with Item Response Theory (IRT) models, and classical test theory (CTT) provide unidimensional ability values, which allow for ranking of students according to their abilities. Cognitive diagnostic assessment (CDA) is aimed at providing formative diagnostic feedback through a fine-grained reporting of learners’ skill mastery. The present study, through application of deterministic input, noisy ‘‘and’’ gate [de la Torre & Douglas, ۲۰۰۴ (DINA) model], investigated the degree to which the items of a high stakes reading comprehension test can provide diagnostically useful information.  Through expert rating, a Q-matrix including five subskills was developed. Using DINA model, student’s skill mastery profiles were produced and issues related to the diagnostic capacity of the reading test were discussed. Meanwhile, this study pedagogically demonstrates the application of the DINA model employing R software.

Keywords:

attribute , Cognitive Diagnostic Model (CDM) , DINA , skill , Q-matrix