Validating an AI-Developed Metaphor Awareness Test (MAT) to Exploring Figurative Language Competence of Kurdish EFL Students

Abstract

Figurative language competence is a cornerstone of higher-level language proficiency, with metaphor and metonymy operating as master tropes. The pervasion of figurative language in natural discourse makes it crucial to the learners to communicate effectively, and those who lack such ability essentially become cut off from enormous amount of input. Tools to measure metaphor awareness must undergo proper statistical and methodological validation. This study adapts and develops a Metaphor Awareness Test (MAT) using generative AI, specifically ChatGPT. After a pilot conducted with 15 students checking validity and reliability, the refined MAT was administered to 92 undergraduate students at pre-intermediate to intermediate proficiency. The administration of the test involved sufficient directions and definitions prior to the test to measure the intended construct (awareness). The instrument serves as the baseline for a subsequent experimental study involving pre- and post-tests for metaphor and metonymy in reading and writing skills, alongside AI-generated textual and multimodal instructional materials. Participants demonstrated a consistent difference in recognizing metaphor and metonymy. They also showed challenges identifying elements of figurative language, with markedly stronger performance on metaphor than on metonymy. The tool achieves psychometric reliability through continuous improvement and adjustments, and the results are relevant for teaching figurative language to Kurdish EFL.