Iranian EFL Writing Assessment: The Agency of Rater or Rating Scale?

The present study explores the practice of Iranian raters in the EFL writing assessment context of the country. For this aim, early in the study a questionnaire developed by the researcher was administered to thirty experienced raters from ten major state universities in Iran. Later, five of the raters were chosen to participate in a follow-up think aloud session aimed to further explore the rating process. Findings of the questionnaire casted doubt on the existence of an explicit rating scale in the context. The following think aloud protocols further revealed that despite the apparent superiority of the raters in the rating process, neither the raters nor the rating scale-as two central components of the performance assessment- had a real agency in the Iranian EFL rating context. Lack of a common rating scale caused the raters to draw on ad hoc rating criteria and raters’ idiosyncratic practices resulted by a missing rater training component created a context in which the construct of writing ability is greatly underrepresented. Along with locating the sources of the issue in both the rating scale and the raters, this study emphasizes that Iranian raters are in urgent need of training and constant monitoring programs to acquire rating expertise over time. As a requirement for the training programs, development of an explicit rating scale is strongly encouraged in the context.

Keywords:

EFL writing assessment , Rating Scale , Rater , Validation , Think aloud protocols