Developing a Model of Analytic Rating Scales to Assess College Students’ L2 Chinese Oral Performance

This study develops a model of analytic rating scales to assess L2 Chinese oral performance. It uses Exploratory Factor Analysis (EFA) to identify a model and employs Confirmative Factor Analysis (CFA) in a separate dataset to test the degree of model fit. The researcher videotaped ten speeches and ACTFL professional raters assessed the oral performances in these samples. The researcher then selected three samples (Samples 1, 2, and 3) to represent the proficiency levels of Novice High, Intermediate High, and Advanced Low. Then, the researcher developed 20 rating items by interviewing ten experienced L2 Chinese teachers and running an EFA. The 20 items were descriptors that Chinese teachers used to assess oral performance in two studies: Study 1 and Study 2. To complete Study 1, the researcher recruited 45 teachers to assess Sample 1 using the 20 items, 62 teachers rated Sample 2, and 49 teachers rated Sample 3. In Study 2, 104 teachers assessed all three samples. The EFA indicated a four-factor model of analytic rating scales: “fluency,” “conceptual understanding,” “communication clarity,” and “communication appropriateness.” In this model, the correlations between these analytic rating scales were relatively high and teachers weighted “fluency” as most important. Together the four scales explained 65.5% of teachers’ holistic judgments of oral performance. The CFA did not show a strong model fit to the data, but the fit was acceptable. This model advances our understanding of the relationship between analytic rating scales and holistic ratings in the context of L2 Chinese. These findings give Chinese teachers with which a reference to assess U.S. college students’ L2 Chinese oral performance.

Developing a Model of Analytic Rating Scales to Assess College Students’ L2 Chinese Oral Performance Keywords:

Analytic rating scales , Factor analysis , Language assessment , L2 Chinese , Oral Performance