WHAT TRAINEE ENGLISH TEACHERS NOTICE IN AI-GENERATED ASSESSMENT TOOLS: FAULT DETECTION IN TWO TESOL COHORTS
DOI:
https://doi.org/10.5281/zenodo.23152721Аннотация
A teacher can now describe a practice activity to a language model and have a working assessment tool a few minutes later. Between that file and the learner, there is one check: the teacher deciding whether to use it. This study examined what trainee English teachers notice in descriptions of such tools. Forty-nine students in two graduate TESOL cohorts (n = 24 and n = 25) each judged seven tools carrying a planted fault. The faults were drawn from an argument-based account of validity and divided into those visible in the content and those residing in the relation between what a tool computes and what it claims to measure. Detection was higher for content faults than for construct faults in both cohorts: 62.5% against 35.4% in the first, and 83.0% against 52.0% in the second. The second cohort detected more than the first on six of the seven faults. The two clearest cohort differences were an incorrect answer key (58.3% against 88.0%) and a four-item claim to a CEFR level. Counting linking words and reporting the count as coherence remained the least frequently detected fault. The pattern supports the claim that language-trained noticing catches language errors and often misses measurement claims. It does not support the stronger claim that construct faults are nearly invisible.Ключевые слова
teacher noticing, language assessment, generative artificial intelligence, validity, assessment literacy, teacher education, CEFRБиблиографические ссылки
Anbreen, T., Özturan, T., Shrestha, P. N., & Maqsood, A. (2026). Human vs. generative artificial intelligence in writing assessment: Investigating feedback alignment, score validity, and teacher agency. Educational Assessment, Evaluation and Accountability. Advance online publication. https://doi.org/10.1007/s11092-026-09486-z
Baxtiyorova, D. I. (2025). Talabalarning ingliz tili ko‘nikmalarini rivojlantirishda sun’iy intellektdan foydalanish metodikasini takomillashtirish [Abstract of doctoral dissertation, Bukhara State University]. https://buxdu.uz/media/aftoreferat/ Durdona_Avtoreferati._20.08.2025docx.pdf
Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102
Chow, M., & Ng, O. (2025). From technology adopters to creators: Leveraging AI-assisted vibe coding to transform clinical teaching and learning. Medical Teacher, 47(12), 1927–1929. https://doi.org/10.1080/0142159X.2025.2488353
Chuang, P.-L., & Yan, X. (2025). Language assessment in the era of generative artificial intelligence: Opportunities, challenges, and future directions. System, 134, Article 103846. https://doi.org/10.1016/j.system.2025.103846
Council of Europe. (2020). Common European Framework of Reference for Languages: Learning, teaching, assessment – Companion volume. Council of Europe Publishing. https://rm.coe.int/common-european-framework-of-reference-for-languages-learning-teaching/16809ea0d4
Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113–
https://doi.org/10.1080/15434303.2011.642041
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
König, J., Santagata, R., Scheiner, T., Adleff, A.-K., Yang, X., & Kaiser, G. (2022). Teacher noticing: A systematic literature review of conceptualizations, research designs, and findings on learning to notice. Educational Research Review, 36, Article 100453. https://doi.org/10.1016/j.edurev.2022.100453
Republic of Uzbekistan. (2024). Resolution of the President of the Republic of Uzbekistan No. PP-358 of October 14, 2024, on approval of the Strategy for the Development of Artificial Intelligence Technologies to 2030. https://lex.uz/ru/ docs/7158606 AKTABGACHA VA AKTABGACHA VA AKTAB TA’LIMI AKTAB TA’LIMI
Sarkar, A., & Drosos, I. (2025). Vibe coding: Programming through conversation with artificial intelligence. In Proceedings of the 36th Annual Conference of the Psychology of Programming Interest Group (PPIG 2025). https://arxiv.org/ abs/2506.23253
Song, Y., Choi, S., Kim, J., Kim, Y., Weisberg, L., & Moon, J. (2026). A guiding framework for K-12 teachers in creating AI-powered learning technologies through vibe coding [Preprint]. arXiv. https://arxiv.org/abs/2607.05406
van Es, E. A., & Sherin, M. G. (2008). Mathematics teachers’ “learning to notice” in the context of a video club. Teaching and Teacher Education, 24(2), 244–276. https://doi.org/10.1016/j.tate.2006.11.005
Woo, D. J., Guo, K., & Yu, Y. (2025). A vibe coding learning design to enhance EFL students’ talking to, through, and about AI [Preprint]. arXiv. https://arxiv.org/abs/2509.08854 AKTABGACHA VA AKTABGACHA VA AKTAB TA’LIMI AKTAB TA’LIMI 13.00.00 Pedagogika fanlari 13.00.01 Pedagogika nazariyasi. Pedagogik ta’limotlar tarixi 13.00.02 Ta’lim va tarbiya nazariyasi va metodikasi (sohalar bo‘yicha) 13.00.03 Maxsus pedagogika 13.00.04 Jismoniy tarbiya va sport mashg‘ulotlari nazariyasi va metodikasi 13.00.05 Kasb-hunar ta’limi nazariyasi va metodikasi 13.00.06 Elektron ta’lim nazariyasi va metodikasi (ta’lim sohalari va bosqichlari bo‘yicha) 13.00.07 Ta’limda menejment 13.00.08 Maktabgacha ta’lim va tarbiya nazariyasi va metodikasi 13.00.09 Ijtimoiy pedagogika 07.00.00 Tarix fanlari 19.00.00 Psixologiya fanlari 01.00.00 Fizika-matematika fanlari 02.00.00 Kimyo fanlari 03.00.00 Biologiya fanlari 09.00.00 Falsafa fanlari 10.00.00 Filologiya fanlari 11.00.00 Geografiya fanlari AKTABGACHA VA AKTABGACHA VA AKTAB TA’LIMI AKTAB TA’LIMI Mas’ul muharrir: Ramzidin Ashurov
Загрузки
Опубликован
Выпуск
Раздел
Лицензия
Copyright (c) 2026 «Maktabgacha va maktab ta’limi» jurnali
Это произведение доступно по лицензии Creative Commons «Attribution» («Атрибуция») 4.0 Всемирная.
Как цитировать
Страницы
Цитирование