Investigating Rater effect on Foreign Language Writing Performance Using Bi-factor Analysis

The current study intends to investigate what influence, if any, rater as a construct irrelevant factor has on foreign language writing test results. Specifically, this study focused on self, peer and teacher raters.The participants were 200 non-native English speakers majoring in English literature and Englishtranslation at an Iranian university. The students were asked to complete a writing task, for completing which they were asked to write on a given subject based on their own experiences. Then, the completedtest task was rated by self, peer and teacher raters based on an analytical scoring rubric. Finally bifactorstructural equation modeling using IMB SPSS AMOS22 software was used to analyze the data. As the next step, fit indices attached to the proposed model were explored. A perfect fit of the model to the data was found, therefore it was revealed that both general (writing) and domain specific factors(self, peer and teacher raters) affect performance in a foreign language writing test. To understandwhether the effect of general factor (students’ writing performance ability) was higher or the effect of domain specific factors (self, peer and teacher raters), item loadings were explored. Each of the threedomain specific factor loadings were higher than factor loadings of the general factor, writing. Theresults indicated that the effect of rater as a construct irrelevant factor on test takers’ foreign language writing performance was much greater than their writing performance ability. The attempt to describe the relationships among the variables certainly extended our understanding ofthe effect of construct irrelevant factors on foreign language writing by concluding that raters affect foreign language writing performance. The current study also provides support for usefulness of SEM approach in exploring issues related to foreign language writing tests. The results of this study will provide recommendations to anyone involved in language assessment, theoretical or practical

Keywords:

Bi-factor , Structural Equation Modeling , Amos , foreign language writing assessment , peer assessment , self-assessment , teacher assessment