Strategic Competence, Task Complexity, and Learner Performance in Computer-delivered Integrated Speaking Test Tasks: A Study of English-as-a-Foreign-Language (EFL) Learners in China

Abstract

Understanding the relationships among test-takers’ strategic competence, test tasks and test performance is a perennial problem in language assessment. Despite numerous research efforts in addressing the problem, little is known about how the intricate relationships play out in the integrated speaking tests, one of the most popular formats in high-stakes tests such as the computer-mediated TOEFL (the Test of English as a Foreign Language). This PhD study aims to fill the research gap in a Chinese EFL context. Underpinned by Bachman and Palmer’s (2010) framework of non-reciprocal language use, the study employed a convergent mixed-methods design involving 616 Chinese EFL students and five Chinese EFL teachers to examine whether Chinese EFL learners’ speaking performance was affected by their strategic competence (i.e., their use of metacognitive strategies) and task complexity. It also investigated if the interaction between strategic competence and task complexity would affect speaking performance. To this end, the student participants answered a questionnaire to report their metacognitive strategy use immediately after they performed four TOEFL-based integrated speaking test tasks. All the participants self-rated the difficulty of the test tasks to measure task complexity. Eight students participated in the subsequent interviews for an in-depth probe into the Chinese EFL learners’ metacognitive strategy use and their perceptions of task complexity. To investigate the intricate relationships, statistical procedures such as one-way repeated-measures MANOVA, structural equation modelling, one-way repeated measures ANOVA, hierarchical linear modelling, and multiple regression were used at different stages of quantitative data analysis. Qualitative coding with content analysis was also conducted. The findings suggest that among the four individual subcomponents of the metacognitive strategies (planning, problem-solving, monitoring and evaluating), only problem-solving reported by the Chinese EFL learners demonstrated substantial variance across the speaking test tasks. By contrast, these learners’ use of interactive metacognitive strategies and their speaking performance were significantly affected by task complexity. Regarding the relationships among the three variables, monitoring as a metacognitive strategy moderated the effect of task complexity on the learners’ speaking performance. In the same vein, prior knowledge was identified as a moderator between the other three task complexity variables (viz., planning time, steps involved and task type) and the learners’ overall perceptions of task complexity involved in the test tasks. Unexpectedly, individual attributes (e.g., motivation and anxiety), though not the focus of the study, were found to mediate the interaction between the learners’ metacognitive strategy use and task complexity. The findings primarily lend some new evidence in support of Bachman and Palmer’s (2010) Strategic Competence Model and their proposed framework of non-reciprocal language use. They also support Robinson’s (2015) Triadic Componential Framework and Kormos’ (2011) Bilingual Speech ProductionModel. Taken together, these findings provide implications for metacognitive scaffolding, syllabus designing and task development for EFL speaking instructions. Moreover, the findings are expected to add empirical evidence for validity arguments for test development: In designing speaking tests, there appears to be a need for taking into consideration task complexity so that the tests can truly assess test-takers’ language ability, as required for meeting the assumptions of test validity and reliability.