The Impact of Chatgpt on Students’ Academic Writing Performance: A Meta-Analytic Study

Jinalyn B. Alvizo

doi:10.55927/jeda.v4i4.381

Authors

Jinalyn B. Alvizo Palompon Institute of Technology Palompon

DOI:

https://doi.org/10.55927/jeda.v4i4.381

Keywords:

Academic Writing, ChatGPT, Meta-Analysis, Tertiary Students, Writing Performance

Abstract

This meta-analytic study examined the impact of ChatGPT-assisted writing instruction on the academic writing performance of tertiary students. Empirical studies published between 2023 and 2025 were systematically reviewed and synthesized to find out the effectiveness of ChatGPT-assisted tool in higher education writing contexts. In this study, 11 studies met the criteria in terms of inclusion, representing diverse academic disciplines and cultural settings. The findings revealed the pooled effect size of 2.18 means it is a large positive effect of ChatGPT integration on students’ writing performance, indicating improvements in grammar, coherence, and content elaboration. In addition, it emphasized that ChatGPT is beneficial supplementary tool for improving students’ writing skills when integrated into instructional design. However, variations in study design, duration, and assessment measures suggest the need for further longitudinal research. Recommendations for higher education institutions include structured implementation of ChatGPT in writing courses, professional development for instructors on AI integration, and emphasis on ethical use to uphold academic integrity.

References

Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of CHATGPT. Contemporary Educational Technology, 15(3). https://doi.org/10.30935/cedtech/13152

Alwasidi, M. A., & Al-Khalifah, K. S. (2025). Assessing the impact of chatgpt on EFL students’ writing productivity and proficiency. Journal of Language Teaching and Research, 16(3), 986–995. https://doi.org/10.17507/jltr.1603.29

Ali, J. K. M., Shamsan, M. A. A., Hezam, T. A., & Mohammed, A. A. Q. (2023). Impact of ChatGPT on learning motivation: Teachers and students’ voices. Journal of English Studies in Arabia Felix, 2(1), 41–49. https://doi.org/10.56540/jesaf.v2i1.51

Bašić, Ž., Banovac, A., Kružić, I. et al. ChatGPT-3.5 as writing assistance in students’ essays. Humanity Soc Sci Commun 10, 750 (2023). https://doi.org/10.1057/s41599-023-02269-7

Bada, S. O., & Olusegun, S. (2015). Constructivism learning theory: A paradigm for teaching and learning. Journal of Research & Method in Education, 5(6), 66–70.

Bonner, E., Lege, R., & Frazier, E. (2023). Large language model-based artificial intelligence in the language classroom: Practical ideas for teaching. Teaching English with Technology, 23(1), 23–41. https://doi.org/10.56297/BKAM1691/WIEO1749

Borenstein, M., Hedges, L., Higgins, J., & Rothstein, H. R. (2009). Computing Effect Sizes for Meta-analysis. Chichester: John Wiley & Sons, Ltd.

Emran, A. Q., Mohammed, M. N., Saeed, H., Abu Keir, M. Y., Alani, Z. N., & Mohammed Ibrahim, F. (2024). Paraphrasing chatgpt answers as a tool to enhance university students’ academic writing skills. 2024 ASU International Conference in Emerging Technologies for Sustainability and Intelligent Systems (ICETSIS), 501–505. https://doi.org/10.1109/icetsis61505.2024.10459386

Farah JC, Ingram S, Spaenlehauer B, Lasne FK-L, Gillet D (2023) Prompting large language models to power educational Chatbots. In: Xie H, Lai, CL., Chen W, Xu G, Popescu, E. (eds.), Advances in web-based learning—ICWL 2023. Springer Nature, pp. 169–188

Hongxia, H., & Razali, A. B. (2025). Impact of chatgpt on English academic writing ability and engagement of Chinese EFL undergraduates. International Journal of Instruction, 18(2), 323–346. https://doi.org/10.29333/iji.2025.18218a

Huong, L. P. H. & Hung, B. P. (2021). Mediation of digital tools in English learning. LEARN Journal: Language Education and Acquisition Research Network, 14(2), 512-528.

Jamshed, M., Abu Saleh Md Manjur Ahmed, Md Sarfaraj, & Wahaj Unnisa Warda. (2024). The impact of CHATGPT on English language learners’ writing skills. International Journal of Interactive Mobile Technologies (iJIM), 18(19), 18–36. https://doi.org/10.3991/ijim.v18i19.50361

Jiang H, Cheng Y, Yang J, et al. (2022) AI-powered chatbot communication with customers: dia-logic interactions, satisfaction, engagement, and customer behavior. Computers in HumanBehavior 134: 107329.

Khalifa, M., & Albadawy, M. (2024). Using artificial intelligence in academic writing and research: An essential productivity tool. Computer Methods and Programs in Biomedicine Update, 5, 100145. https://doi.org/10.1016/j.cmpbup.2024.100145

Khampusaen, D. (2024). The impact of ChatGPT on academic writing skills and knowledge: An investigation of its use in argumentative essays. LEARN Journal: Language Education and Acquisition Research Network. 18(1), 963-988. https://doi.org/10.70730/PGCQ9242

Kurteš, S., & Kopytowska, M. (2014). Communicating identities in daily interaction: Theory, practice, pedagogy. Lodz Papers in Pragmatics, 10(1), 1-18. https://doi.org/10.1515/lpp-2014-0001

Lantolf, J. P. (2006). Sociocultural theory and L2: State of the art. Studies in Second Language Acquisition, 28(1) 67-109. https://doi.org/10.1017/S0272263106060037

Mahapatra, S. (2024). Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study. Smart Learn. Environ. 11, 9. https://doi.org/10.1186/s40561-024-00295-9

Nakavachara, V., Potipiti, T., & Chaiwat, T. (2024). Experimenting with Generative AI: Does chatgpt really increase everyone’s productivity? SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4746770

Orrù, G., Piarulli, A., Conversano, C., & Gemignani, A. (2023). Human-like problem-solving abilities in large language models using chatgpt. Frontiers in Artificial Intelligence, 6. https://doi.org/10.3389/frai.2023.1199350

Ozdere, M. (2025). Ai in academic writing: Assessing the effectiveness, grading consistency, and student perspectives of chatgpt and you.com for EFL students. International Journal of Technology in Education, 8(1), 123–154. https://doi.org/10.46328/ijte.1001

Paas, F., & Sweller, J. (2014). Implications of Cognitive Load Theory for Multimedia Learning. In The Cambridge Handbook of Multimedia Learning (Vol. 27, pp. 27-42). Cambridge University Press.

Qazi, M. H., Munir, T., Zaigham, M. S., & Mughal, U. A. (2025). The impact of AI-assisted self-regulated learning, specifically ChatGPT, on students’ engagement and writing skill enhancement. Journal of Applied Linguistics and TESOL, 7(1).

Roumeliotis, K. I., & Tselikas, N. D. (2023). ChatGPT and Open-AI Models: A Preliminary Review. Future Internet, 15, Article 192.

https://doi.org/10.3390/fi15060192

Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1), 342–363. https://doi.org/10.37074/jalt.2023.6.1.9

Shi, H., Chai, C. S., Zhou, S., & Aubrey, S. (2025). Comparing the effects of CHATGPT and automated writing evaluation on students’ writing and ideal L2 writing self. Computer Assisted Language Learning, 1–28. https://doi.org/10.1080/09588221.2025.2454541

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.