Machine translation of phraseological units from English to Croatian

master's thesis

• Title (croatian): Strojni prijevod frazeoloških jedinica s engleskog jezika na hrvatski jezik
• Author: Sandra Razum
• Mentor: Mateusz-Milan Stanojević (mentor)
• Mentor: Nataša Pavlović (komentor)
• Committee member: Snježana Veselica Majhut (predsjednik povjerenstva)
• Granter: University of Zagreb; Faculty of Humanities and Social Sciences; (Department of English language and literature); Zagreb
• Defense date and country: 2024-11-04, Croatia
• Scientific / art field, discipline and subdiscipline: HUMANISTIC SCIENCES; Philology; Anglistics

Abstract

The reliability of machine translation (MT) has increased greatly over the last several years with the rapid development of neural MT systems using deep learning and, more recently, large language models which perform MT among their other tasks. While studies have shown that neural MT may still provide poor quality output for low-resource languages and have trouble conveying the meaning of non-literal language, it is widely used within the translation industry, translator education, and in the everyday lives of end-users. The recent introduction of large language models, such as ChatGPT, has opened new avenues for automated translation which have not yet been fully explored. This study aimed to compare the quality of translation of phraseological units (PUs) by Google Translate and ChatGPT-3.5, using human evaluation. The two systems achieved similar results in accuracy and usage of PUs in their translations, although ChatGPT was marginally more successful in accuracy. They also both accurately translated most PUs using a PU and shared word-for-word translation as their most common error. Notable differences lay in the correlation found between Google Translate’s translation accuracy and the PUs’ absolute frequency in the English SkELL corpus as well as the appearance of hallucinations in ChatGPT.

Abstract (croatian)

Razina pouzdanosti strojnih prijevoda značajno je porasla u proteklih nekoliko godina zbog naglog razvoja dubokog učenja neuralnih strojnih prevoditelja i novih velikih jezičnih modela koji se, između ostaloga, mogu koristiti kao strojni prevoditelji. Istraživanja su pokazala da u jezicima s ograničenim resursima strojni prijevodi i dalje mogu biti lošije kvalitete te da strojni prevoditelji imaju poteškoće sa prevođenjem nedoslovnih značenja. Unatoč tome, strojni se prijevodi često upotrebljavaju u prevoditeljskoj industriji, u obrazovanju prevoditelja te u svakodnevnici krajnjih korisnika. Nedavno uvođenje velikih jezičnih modela poput ChatGPT-a u te procese otvorilo je nove mogućnosti za automatizirano prevođenje koje se još nisu u potpunosti istražile. Cilj je ovog istraživanja bio usporediti kvalitetu prijevoda frazeoloških jedinica sustava Google Translate i ChatGPT-3.5 uz korištenje ljudske evaluacije. Sustavi su postigli sličan rezultat što se tiče točnosti i broja korištenih frazeoloških jedinica u prijevodima, iako je ChatGPT bio neznatno uspješniji u točnosti prijevoda. Oba su sustava točno prevela većinu frazeoloških jedinica pomoću frazeoloških jedinica, a doslovan prijevod bio im je najčešća greška. Značajne razlike između sustava bile su u korelaciji između točnosti prijevoda Google Translatea i apsolutne frekvencije frazeoloških jedinica u engleskom SkELL korpusu te u pojavi halucinacija kod ChatGPT-a.

• Keywords: neural machine translation; large language model; phraseology; English; Croatian; corpora; human evaluation
• Keywords (croatian): neuronski strojni prijevod; veliki jezični model; frazeologija; engleski; hrvatski; korpus; ljudska evaluacija
• Language: english
• URN:NBN: urn:nbn:hr:131:950907
• Study programme: Title: English Language and Literature (double major); specializations in: Linguistics, Literature and Culture, Translation, TEFL; Course: Translation; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra engleskog jezika i književnosti (magistar/magistra engleskog jezika i književnosti)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Open access
• Created on: 2024-11-07 13:34:16