Evaluating the Pragmatic Competence of Artificial Intelligence in the Kurdish Language: »ChatGPT, Gemini, and Claude« as a Case Study

Abstract

The rapid advancements of Large Language Models »LLMs «have achieved remarkable success in language analysis; however, their ability to comprehend implicit meanings and pragmatic aspects in low-resource languages, such as Kurdish, still requires rigorous scientific evaluation and novel research. This study aims to evaluate and compare the pragmatic competence of three advanced artificial intelligence models» ChatGPT, Gemini, and Claude« in perceiving the indirect intentions and implicit meanings of speakers within Kurdish texts. Methodologically, the research employs a descriptive-analytical and comparative approach. Data were collected by administering 12 specifically designed linguistic scenarios to all three models using a direct, zero-shot prompting technique. The analyses were conducted within the framework of the Central Kurdish dialect (Sorani), drawing upon both "Speech Act Theory" and "Grice's Cooperative Principle." The study operates on the hypothesis that while these models demonstrate a proficient understanding of general Kurdish texts, they encounter significant challenges in grasping pragmatic intentions and correctly employing Kurdish academic linguistic terminology. Furthermore, it is anticipated that ChatGPT and Gemini, owing to their larger scale and broader exposure to diverse datasets, will yield more precise analyses compared to Claude This study relies on the “praise-analysis and comparison method”, in which through the design of a targeted experiment, consisting of 12 examples of linguistic scenarios, questions were asked directly to the models using zero-shot prompting technique They have been collected, classified and compared statistically and linguistically in order to reach scientific conclusions.