The Effect of AI-Assisted Learning on EFL Writing Proficiency: Quasi-Experimental and Cluster Analysis

Background/purpose. The study investigates whether the AI-assisted learning leads to a significant difference in student scores across the measured writing skills, and it also provides valuable insights for teachers and students on utilizing ChatGPT effectively. Specifically, the study applies a quasi-experimental design to assess five core dimensions of writing proficiency: content, organization, vocabulary, language use, and mechanics. Materials/methods. The research was conducted with a sample of 10 high school students in Dhahran, who were selected through convenience sampling. Over a three-week period, the participants engaged in sessions, practicing how to write while using ChatGPT as a supplementary aid simultaneously. The AI tool provided real-time feedback on grammar, vocabulary, and coherence. The study utilized pre-test and post-test assessments to measure improvements in five specific competencies, namely, content, organization, vocabulary, language use, and mechanics. Participants were at CEFR levels A2–B1. Writing performance was evaluated using Jacobs et al.’s (1981) analytical scoring rubric. Statistical analyses included the Wilcoxon Signed Rank Test to assess significance and Two-Step Cluster Analysis to examine learner response patterns. Results. The results revealed a substantial increase in scores for content, language use, and vocabulary from pre-test to post-test phases. However, the advancement in scores for organization and mechanics was minuscule. The findings suggest that incorporating AI tools for writing, such as ChatGPT, into EFL instruction can augment certain aspects of writing proficiency. Statistically significant gains were observed in content (p < .01), language use (p < .05), and vocabulary (p ≈ .05), while organization and mechanics did not show significant changes. Cluster analysis identified two distinct student groups, with one demonstrating stronger performance gains. Conclusion. The increases in the average scores for content, vocabulary, language use, and the total scores indicate that the AI-assisted training was practical in helping EFLs enhance their writing proficiency. Overall, AI-assisted learning shows much promise in improving the writing proficiency of EFL students, but continued research is imperative to explore its full potential while addressing its limitations concurrently. Future research should also consider potential confounding variables, such as age, gender, first language, and cultural background. Accommodating the above factors will ensure accurate and comprehensive results.