The Effects of Rater Training on Raters’ Severity and Bias in Second Language Writing Assessment

The assessment of writing has always been threatened due to raters’ biasedness. There is evidence that rater training can be effective in eliminating extreme differences in raters’ severity, and increasing the self consistency of raters by reducing individual biases of raters (Weigle, ۱۹۹۴a). However, there is little research documenting the exact as well as the amount of the effectiveness of training in reducing this biasedness. The purpose of this study is to investigate how judgments of raters are biased towards certain criteria before and after the training program in assessing second language essay compositions. ۱۲ EFL raters scored ۴۰ pre-rated benchmark essay compositions rated by an authorized IELTS trainer. These essay compositions were scored before and after the training program. The results show that most raters were able to modify their scoring, resulting in greater intergroup consistency and reduced biasedness and severity/leniency. Raters who were identified as being highly severe/lenient and biased in particular categories of the rating scale were no longer biased after training.

Keywords:

Rater training , Writing assessment , Raters’ Biasedness , Raters’ severity , Raters’ leniency