Improvement of Design of Multiple Choice Questions in Annual Residency Exams by Giving Feedback

Background & Objective: The aim of this study was to survey the effect of presenting feedback on improvement of multiple choice questions (MCQ) design in annual residency exams Methods: In this semiexperimental study the structure and taxonomy in MCQs of annual residency exams in ۲۰۰۹ were analyzed through Millmans checklist Based on this data an interventional feedback on strengths and weaknesses was presented to question designers who had experience of attending MCQ workshops and designing questions in the past The principles of giving technical feedback in medical education such as objectivity right timing and being direct respectful and without judgment were considered After the annual residency exam in ۲۰۱۰ the questions of designers in both years were compared in respect to Millmans principals and taxonomy before and after the intervention Results: From ۲۵۰۰ questions ۲۰۳۵ questions were entered into the study ۶۵۲% of the questions developed in ۲۰۰۹ had no structural flaw this increased to ۸۲۸% after the intervention which was a significant difference (P < ۰۰۰۱) Moreover there was a significant increase in the percentage of questions with a high taxonomy from ۳۸% in ۲۰۰۹ to ۵۳۱% in ۲۰۱۰ (P < ۰۰۰۱) Conclusion: Giving interventional feedback based on its techniques in medical education to experienced designers of MCQs can affect the improvement of designing quality of questions

Keywords:

Multiple choice questions , Residency , medical education , Taxonomy , Millman's principles , Feedback