Development of an Instrument for the Online Evaluation of Orthodontic-Treated Cases: A Pilot Study

Background: It is necessary to create a sound method for the optimal evaluation of treated cases of orthodontic board exam candidates. This cross-sectional study aimed to design and validate a new method for the national orthodontic board exam using a web-based program. Methods: Complete documents of ۱۰ patients randomly selected from a pool of the previously presented cases at the national board examination were entered into a web-based program called "the Orthoboard". The documents were arranged according to the European Board of Orthodontist Standards, and ۱۵ related questions were asked based on the index of complexity, outcome, and need index and the American Board of Orthodontics evaluation standards. A customized grading system was used for the finalized questionnaire. They were asked to be evaluated by ۱۰ orthodontists (۵ with less and ۵ with more than ۱۰ years of experience). The content validity of the questionnaire was analyzed by the content validity index and content validity ratio. The reliability of this questionnaire was measured using the Kappa statistical test. Results: Most evaluators do not consider it necessary to ask questions about the diagnostic resume, problem list, treatment plan, and treatment resume. Crowding after treatment and cross-biting before and after treatment had the best interexaminer reliability. The least agreement among examiners was between the pre-treatment aesthetic index and the pre-treatment buccal segment relationship. Conclusion: Inter-examiner reliability was lower than expected, indicating that the orthodontic board test scoring is too subjective. The addition of a ۳-dimensional cast is recommended for better objective evaluation.