Developing a Model of Analytic Rating Scales to Assess College Students’ L۲ Chinese Oral Performance

This study develops a model of analytic rating scales to assess L۲ Chinese oral performance. It uses Exploratory Factor Analysis (EFA) to identify a model and employs Confirmative Factor Analysis (CFA) in a separate dataset to test the degree of model fit. The researcher videotaped ten speeches and ACTFL professional raters assessed the oral performances in these samples. The researcher then selected three samples (Samples ۱, ۲, and ۳) to represent the proficiency levels of Novice High, Intermediate High, and Advanced Low. Then, the researcher developed ۲۰ rating items by interviewing ten experienced L۲ Chinese teachers and running an EFA. The ۲۰ items were descriptors that Chinese teachers used to assess oral performance in two studies: Study ۱ and Study ۲. To complete Study ۱, the researcher recruited ۴۵ teachers to assess Sample ۱ using the ۲۰ items, ۶۲ teachers rated Sample ۲, and ۴۹ teachers rated Sample ۳. In Study ۲, ۱۰۴ teachers assessed all three samples. The EFA indicated a four-factor model of analytic rating scales: “fluency,” “conceptual understanding,” “communication clarity,” and “communication appropriateness.” In this model, the correlations between these analytic rating scales were relatively high and teachers weighted “fluency” as most important. Together the four scales explained ۶۵.۵% of teachers’ holistic judgments of oral performance. The CFA did not show a strong model fit to the data, but the fit was acceptable. This model advances our understanding of the relationship between analytic rating scales and holistic ratings in the context of L۲ Chinese. These findings give Chinese teachers with which a reference to assess U.S. college students’ L۲ Chinese oral performance.

Keywords:

Analytic rating scales , Factor analysis , Language assessment , L۲ Chinese , Oral Performance