Bibliometric analysis and visualization of translation assessment: Research theme, evolution and hotspots
Abstract
Translation assessment, referring to evaluating various aspects related to translation, is crucial to the improvement of translation competence and quality. This study, taking valid papers on translation assessment published in WoS core collection from 2000 to 2022 as research samples, visualizes and reviews the research theme, research evolutions and emerging hot topics of translation assessment through bibliometric analysis. The number of annual publications shows that despite the general growth, the number of articles on translation assessment fluctuates from year to year. By Cite Space-based analysis of keywords co-occurrence, keyword clustering, time zone and burst detection, it finds that translation assessment mainly covers five themes, namely translation competence, translation quality, machine translation, translation teaching and training, and others. The overall evolution has been a gradual shift from concepts, metrics and evaluations of topics related to translation assessment into a topic of in-depth and interdisciplinary study. The hotspots in recent years include translation competence acquisition, neural machine translation and translation quality estimation. The findings reveal research evolutions and hot spots, propose implications for further translation assessment research and provide references for scholars who are interested in this topic.
References
[1] Adom, D., Adu-Mensah, J., & Dake, D. A. (2020). Test, measurement, and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education (IJERE), 9(1), 109. https://doi.org/10.11591/ijere.v9i1.20457
[2] Airasian, P.W. (1997). Classroom assessment, 3rd ed. McGraw-Hill.
[3] Akulina, K. V., & Tikhonova, E. V. (2021). Application of the translation quality assessment system in the training of linguist-translators. Yazyk i Kul’tura, 56, 129–145. https://doi.org/10.17223/19996195/56/8
[4] Albir, A. H. (2007). Competence-based Curriculum Design for Training Translators. The Interpreter and Translator Trainer, 1(2), 163–195. https://doi.org/10.1080/1750399x.2007.10798757
[5] Albir, P. Group. A. H., Galán-Mañas, A., Kuznik, A., Olalla-Soler, C., Rodríguez-Inés, P., & Romero, L. (2020). Translation competence acquisition. Design and results of the PACTE group’s experimental research. The Interpreter and Translator Trainer, 14(2), 95–233. https://doi.org/10.1080/1750399x.2020.1732601
[6] Alos, J. (2015). Explicating the implicit: an empirical investigation into pragmatic competence in translator training. The Interpreter and Translator Trainer, 9(3), 287–305. https://doi.org/10.1080/1750399x.2015.1100398
[7] al-Qinai, J. (2002). Translation Quality Assessment. Strategies, Parametres and Procedures. Meta, 45(3), 497–519. https://doi.org/10.7202/001878ar
[8] Angelone, E. (2020). The impact of screen recording as a diagnostic process protocol on inter-rater consistency in translation assessment Erik Angelone, Kent State University. Journal of Specialised Translation, 34, 32-50.
[9] Angelone, E. (2021). Broadening the scope of error categories in translation assessment through screen recording. Across Languages and Cultures, 22(2), 143–157. https://doi.org/10.1556/084.2021.00021
[10] Arango-Keeth, F., & Koby, G. S. (2003). Assessing assessment. Beyond the Ivory Tower, 117–134. https://doi.org/10.1075/ata.xii.09ara
[11] Bai, L., Feng, L., & Yan, M. (2018). Constructs and principles of the Chinese English Translation Proficiency Rating Scale. Modem Foreign Language, 41(1), 101-110.
[12] Baniata, L. H., Park, S., & Park, S.-B. (2018). A Multitask-Based Neural Machine Translation Model with Part-of-Speech Tags Integration for Arabic Dialects. Applied Sciences, 8(12), 2502. https://doi.org/10.3390/app8122502
[13] Beeby, A., Castillo, L., Fox, O., Galán Mañas, A., Hurtado Albir, A., Kuznik, A., Massana, G., Neunzig, W., Olalla, Ch., Rodríguez Inés, P., & Romero, L. (2015). Results of PACTE’s Experimental Research on the Acquisition of Translation Competence. Translation Spaces, 4(1), 29–53. Portico. https://doi.org/10.1075/ts.4.1.02bee
[14] Bowker, L. (2002). Towards a Methodology for a Corpus-Based Approach to Translation Evaluation. Meta, 46(2), 345–364. https://doi.org/10.7202/002135ar
[15] Brunette, L. (2000). Towards a Terminology for Translation Quality Assessment. The Translator, 6(2), 169–182. https://doi.org/10.1080/13556509.2000.10799064
[16] Brunette, L. (2000). Towards a Terminology for Translation Quality Assessment. The Translator, 6(2), 169–182. https://doi.org/10.1080/13556509.2000.10799064
[17] Burchardt, A., Lommel, A., Bywood, L., Harris, K., & Popović, M. (2016). Machine translation quality in an audiovisual context. Target, 28(2), 206–221. Portico. https://doi.org/10.1075/target.28.2.03bur
[18] Cao, D. (2014). Teaching and learning legal translation. Semiotica, 2014(201). https://doi.org/10.1515/sem-2014-0022
[19] Carl, M., & Báez, M. C. T. (2019). Machine translation errors and the translation process: A study across different languages. Journal of Specialised Translation, 31, 107-132.
[20] Castilho, S., & O’Brien, S. (2018). Acceptability of machine-translated content: A multi-language evaluation by translators and end-users. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 16. https://doi.org/10.52034/lanstts.v16i0.430
[21] Chakhachiro, R. (2005). Revision for quality. Perspectives, 13(3), 255–238. https://doi.org/10.1080/09076760508668994
[22] Chapelle, C. A., & Brindley, G. (2010). What is language assessment. 247-267.
[23] Chatzikoumi, E. (2019). How to evaluate machine translation: A review of automated and human metrics. Natural Language Engineering, 26(2), 137–161. https://doi.org/10.1017/s1351324919000469
[24] Che, S., Kamphuis, P., Zhang, S., Zhao, X., & Kim, J. H. (2022). A Visualization Analysis of Crisis and Risk Communication Research Using CiteSpace. International Journal of Environmental Research and Public Health, 19(5), 2923. https://doi.org/10.3390/ijerph19052923
[25] Colina, S. (2008). Translation Quality Evaluation. The Translator, 14(1), 97–134. https://doi.org/10.1080/13556509.2008.10799251
[26] Comelles, E., & Atserias, J. (2018). VERTa: a linguistic approach to automatic machine translation evaluation. Language Resources and Evaluation, 53(1), 57–86. https://doi.org/10.1007/s10579-018-9430-2
[27] Cortez-Godínez, J. (2019). Influencia del nivel socioeconómico en la adquisición de la Competencia Traductora. Sendebar, 30(0). https://doi.org/10.30827/sendebar.v30i0.6594
[28] Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L. (2017). Identifying the Machine Translation Error Types with the Greatest Impact on Post-editing Effort. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.01282
[29] De Sutter, G., Cappelle, B., De Clercq, O., Loock, R., & Plevoets, K. (2018). Towards a Corpus-based, Statistical Approach to Translation Quality: Measuring and Visualizing Linguistic Deviance in Student Translations. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 16. https://doi.org/10.52034/lanstts.v16i0.440
[30] Diao, B.C., Adnan, N., Harji, M. B., Ravindran, L. (2022). Evolution and hotspots of peer instruction: a visualized analysis using CiteSpace. Education and Information Technologies. 1-18.
[31] Ding, Z.Y., Zang, H.F., Zhang, Q., Miao, G.M. & Chi, Y.H. Automatic machine translation evaluation based on sentence structure information. Paper presented at the 2009 International Conference on Asian Language Processing.
[32] Du, W., & Saeheaw, T. (2020). Errors in translation: A case study of Chinese into English translation. Language Learning in Higher Education, 10(1), 129–142. https://doi.org/10.1515/cercles-2020-2009
[33] Dugonik, J., Bošković, B., Brest, J., & Sepesy Maučec, M. (2019). Improving Statistical Machine Translation Quality Using Differential Evolution. Informatica, 30(4), 629–645. https://doi.org/10.15388/informatica.2019.222
[34] Dussart, A. (2005). Faux sens, contresens, non-sens… un faux débat ? Meta, 50(1), 107–119. https://doi.org/10.7202/010661ar
[35] Ellegaard, O., & Wallin, J. A. (2015). The bibliometric analysis of scholarly production: How great is the impact? Scientometrics, 105(3), 1809–1831. https://doi.org/10.1007/s11192-015-1645-z
[36] Erten, C., Harding, P. J., Kobourov, S. G., Wampler, K., & Yee, G. (2004). Exploring the computing literature using temporal graph visualization SPIE Proceedings. https://doi.org/10.1117/12.539245
[37] Finch, A., Yasuda, K., Okuma, H., Sumita, E., Nakamura, S. (2011). A Bayesian Model of Transliteration and Its Human Evaluation When Integrated into a Machine Translation System. IEICE Transactions on Information and Systems, E94-D(10), 1889–1900. https://doi.org/10.1587/transinf.e94.d.1889
[38] Fomicheva, M., & Specia, L. (2019). Taking MT Evaluation Metrics to Extremes: Beyond Correlation with Human Judgments. Computational Linguistics, 45(3), 515–558. https://doi.org/10.1162/coli_a_00356
[39] Gambier, Y., & van Doorslaer, L. (Eds.). (2007). The Metalanguage of Translation. Target, 19(2). Portico. https://doi.org/10.1075/target.19.2
[40] Garcia, I. (2010). Is machine translation ready yet? Target, 22(1), 7–21. Portico. https://doi.org/10.1075/target.22.1.02gar
[41] Gile, D. (2000). The History of Research into Conference Interpreting. Target, 12(2), 297–321. Portico. https://doi.org/10.1075/target.12.2.07gil
[42] Göpferich, S. (2013). Translation competence. Target, 25(1), 61–76. Portico. https://doi.org/10.1075/target.25.1.06goe
[43] Guo, Y.-M., Huang, Z.-L., Guo, J., Li, H., Guo, X.-R., & Nkeli, M. J. (2019). Bibliometric Analysis on Smart Cities Research. Sustainability, 11(13), 3606. https://doi.org/10.3390/su11133606
[44] Guzmán, F., Joty, S., Màrquez, L., & Nakov, P. (2017). Machine translation evaluation with neural networks. Computer Speech & Language, 45, 180–200. https://doi.org/10.1016/j.csl.2016.12.005
[45] Hague, D. R. (2015). Empathic accuracy and translator training: helping students imagine other minds. The Interpreter and Translator Trainer, 9(1), 20–29. https://doi.org/10.1080/1750399x.2015.1010359
[46] Hague, D., Melby, A., & Zheng, W. (2011). Surveying Translation Quality Assessment. The Interpreter and Translator Trainer, 5(2), 243–267. https://doi.org/10.1080/13556509.2011.10798820
[47] Hajmohammadi, A. (2005). TRANSLATION EVALUATION IN A NEWS AGENCY. Perspectives, 13(3), 215–224. https://doi.org/10.1080/09076760508668993
[48] Han, C. (2020). Translation quality assessment: a critical methodological review. The Translator, 26(3), 257–273. https://doi.org/10.1080/13556509.2020.1834751
[49] Han, C., Hu, B., Fan, Q., Duan, J., & Li, X. (2022). Using computerised comparative judgement to assess translation. Across Languages and Cultures, 23(1), 56–74. https://doi.org/10.1556/084.2022.00001
[50] Han, Y., & Meng, S. (2022). Machine English Translation Evaluation System Based on BP Neural Network Algorithm. Computational Intelligence and Neuroscience, 2022, 1–10. https://doi.org/10.1155/2022/4974579
[51] Hassan, H., Hearne, M., Way, A., & Sima’an, K. (2006). Syntactic phrase-based statistical machine translation. Paper presented at the 2006 IEEE Spoken Language Technology Workshop. https://doi.org/10.1109/slt.2006.326799
[52] House, J. (2002). Translation Quality Assessment: Linguistic Description versus Social Evaluation. Meta, 46(2), 243–257. https://doi.org/10.7202/003141ar
[53] Huang, Q., & Xin, X. (2020). A bibliometric analysis of translation criticism studies and its implications. Perspectives, 28(5), 737–755. https://doi.org/10.1080/0907676x.2020.1740750
[54] Janmaijaya, M., Shukla, A., Abraham, A., & Muhuri, P. (2018). A Scientometric Study of Neurocomputing Publications (1992–2018): An Aerial Overview of Intrinsic Structure. Publications, 6(3), 32. https://doi.org/10.3390/publications6030032
[55] Jiang, C. (2010). Quality assessment for the translation of museum texts: application of a systemic functional model. Perspectives, 18(2), 109–126. https://doi.org/10.1080/09076761003678734
[56] Károly, A. (2014). Translation in foreign language teaching: A case study from a functional perspective. Linguistics and Education, 25, 90–107. https://doi.org/10.1016/j.linged.2013.09.009
[57] Karoubi, B. (2016). Translation quality assessment demystified. Babel, 62(2), 253–277. Portico. https://doi.org/10.1075/babel.62.2.05kar
[58] Kasperavičienė, R., & Horbačauskienė, J. (2020). Self-revision and other-revision as part of translation competence in translator training. Journal of Language and Cultural Education, 8(1), 117–133. https://doi.org/10.2478/jolace-2020-0007
[59] Ke, X.H. (2009). An Automatic Translation Evaluation System Based on Semantic Similarities and Fuzzy Neartude. Paper presented at the 2009 International Conference on Environmental Science and Information Application Technology.
[60] Kiraly, D. (2000). A Social Constructivist Approach to Translator Education: Empowerment from Theory to Practice. St Jerome Publishing.
[61] Kit, C., & Wong, T. M. (2008). Comparative evaluation of online machine translation systems with legal texts. Law Library Journal, 100(2), 299-321.
[62] Korol, T. (2020). Translation assessment: Is there anything to be tested objectively? Advanced Education, 15, 32-40. https://doi.org/10.20535/2410-8286.200720
[63] Korol, T. (2021). Digital teacher feedback as a translation assessment tool. Journal of Teaching English for Specific and Academic Purposes, 575. https://doi.org/10.22190/jtesap2104575k
[64] Kudo, T. (2018). Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/p18-1007
[65] Lee, Y. O., & Ronowicz, E. (2014). The development of an error typology to assess translation from English into Korean in class. Babel, 60(1), 35–51. Portico. https://doi.org/10.1075/babel.60.1.03lee
[66] Li, D. (2006). Making Translation Testing More Teaching-oriented: A Case Study of Translation Testing in China. Meta, 51(1), 72–88. https://doi.org/10.7202/012994ar
[67] Liu, L., Fujita, A., Utiyama, M., Finch, A., & Sumita, E. (2017). Translation Quality Estimation Using Only Bilingual Corpora. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(9), 1762–1772. https://doi.org/10.1109/taslp.2017.2716195
[68] Lommel, A., Uszkoreit, H., & Burchardt, A. (2014). Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics. Tradumàtica Tecnologies de La Traducció, 12, 455–463. https://doi.org/10.5565/rev/tradumatica.77
[69] Maier, C. (2000). Introduction. The Translator, 6(2), 137–148. https://doi.org/10.1080/13556509.2000.10799062
[70] Maier, C. (2014). Evaluation and Translation. Routledge. https://doi.org/10.4324/9781315759999
[71] Martínez Melis, N., & Hurtado Albir, A. (2002). Assessment In Translation Studies: Research Needs. Meta, 46(2), 272–287. https://doi.org/10.7202/003624ar
[72] McAlester, G. (1999). “The Source Text in Translation Assessment”. In Words, Words, Words. The Translator and the Language Learner. In: Anderman G, Rogers, M. (editors). pp. 169-178. Clevedon, Philadelphia and Adelaide: Multilingual Matters.
[73] Mei, H., & Chen, H. (2022). Assessing Students’ Translation Competence: Integrating China’s Standards of English With Cognitive Diagnostic Assessment Approaches. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.872025
[74] Mellinger, C. D. (2018). Re-thinking translation quality. Target, 310–331. https://doi.org/10.1075/target.16104.mel
[75] Moorkens, J. (2018). What to expect from Neural Machine Translation: a practical in-class translation evaluation exercise. The Interpreter and Translator Trainer, 12(4), 375–387. https://doi.org/10.1080/1750399x.2018.1501639
[76] Munday, J. (2016). Introducing Translation Studies. Routledge. https://doi.org/10.4324/9781315691862
[77] Nida, E., & Taber, C. (2003). The Theory and Practice of Translation. https://doi.org/10.1163/9789004496330
[78] Nikolaeva, S., & Korol, T. (2021). Prospective Philologists’ translation assessment triangulation: screen video recording and think aloud protocol combination. Advanced Education, 8(18), 30–41. LOCKSS. https://doi.org/10.20535/2410-8286.228550
[79] Olawumi, T. O., & Chan, D. W. M. (2018). A scientometric review of global research on sustainability and sustainable development. Journal of Cleaner Production, 183, 231–250. https://doi.org/10.1016/j.jclepro.2018.02.162
[80] Orozco, M., & Hurtado Albir, A. (2004). Measuring Translation Competence Acquisition. Meta, 47(3), 375–402. https://doi.org/10.7202/008022ar
[81] PACET (2019). Evolution of the Efficacy of the Translation Process in Translation Competence Acquisition. Meta: Journal Des Traducteurs, 64(1), 242. https://doi.org/10.7202/1065336ar
[82] PACTE (2005). Investigating translation competence: conceptual and methodological issues. Volet Traduction, 50(2), 609–619. https://doi.org/10.7202/011004ar
[83] Palumbo, G. (2009). Key terms in translation studies: Bloomsbury Publishing.
[84] Pan, W., Jian, L., & Liu, T. (2019). Grey system theory trends from 1991 to 2018: a bibliometric analysis and visualization. Scientometrics, 121(3), 1407–1434. https://doi.org/10.1007/s11192-019-03256-z
[85] Pollack, J., & Adler, D. (2015). Emergent trends and passing fads in project management research: A scientometric analysis of changes in the field. International Journal of Project Management, 33(1), 236–248. https://doi.org/10.1016/j.ijproman.2014.04.011
[86] Popescu, T. (2013). A Corpus-based Approach to Translation Error Analysis. a Case-study of Romanian EFL Learners. Procedia - Social and Behavioral Sciences, 83, 242–247. https://doi.org/10.1016/j.sbspro.2013.06.048
[87] Pranckutė, R. (2021). Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications, 9(1), 12. https://doi.org/10.3390/publications9010012
[88] Pym, A. (2004). Redefining Translation Competence in an Electronic Age. In Defence of a Minimalist Approach. Meta, 48(4), 481–497. https://doi.org/10.7202/008533ar
[89] Pym, A. (2013). Research skills in translation studies: What we need training in. Across Languages and Cultures, 14(1), 1–14. https://doi.org/10.1556/acr.14.2013.1.1
[90] Sadat, F., & Habash, N. (2006). Combination of Arabic preprocessing schemes for statistical machine translation. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL - ACL ’06. https://doi.org/10.3115/1220175.1220176
[91] Schwenk, H., & Esteve, Y. (2008). Data selection and smoothing in an open-source system for the 2008 NIST machine translation evaluation. Interspeech 2008. https://doi.org/10.21437/interspeech.2008-676
[92] Şerban, A., & Mureşan, V. (2019). Register and Translation Competence Acquisition. Investigating Student Attitudes. Studia Universitatis Babeș-Bolyai Philologia, 64(4), 213–230. https://doi.org/10.24193/subbphilo.2019.4.13
[93] Si, Y.H., Shi, J.G., Chen, J.D., Wu, G.D., & Wang, H.M., (2019). Reviewing sustainable development research in China from the perspective of China Population, Resources and Environment. China population, resources and environment, 29(7), 166-176.
[94] Su, W. (2021). Understanding rubric use in peer assessment of translation. Perspectives, 30(1), 71–85. https://doi.org/10.1080/0907676x.2020.1862260
[95] Tingting, L., & Mengyu, X. (2019). Analysis and evaluation on the quality of news text machine translation based on neural network. Multimedia Tools and Applications, 79(23–24), 17015–17026. https://doi.org/10.1007/s11042-019-7532-5
[96] Tymoczko, M. (2006). Trajectories of Research in Translation Studies. Séances Plénières, 50(4), 1082–1097. https://doi.org/10.7202/012062ar
[97] Waddington, C. (2002). Different Methods of Evaluating Student Translations: The Question of Validity. Meta, 46(2), 311–325. https://doi.org/10.7202/004583ar
[98] Wang, H., Wu, H., He, Z., Huang, L., & Church, K. W. (2022). Progress in Machine Translation. Engineering, 18, 143–153. https://doi.org/10.1016/j.eng.2021.03.023
[99] Wang, L., & Wang, X. (2020). How to evaluate literary translations in the classroom context: through error analysis or scale-based method? Current Trends in Translation Teaching and Learning E, 7, 276-313.
[100] Williams, M. (2002). The Application of Argumentation Theory to Translation Quality Assessment. Meta, 46(2), 326–344. https://doi.org/10.7202/004605ar
[101] Xiao, W.Q. (2012). Study on the Undergraduate Translation Major Testing. People’s Publishing House.
[102] Yang, M., Zhu, J., Li, J., Wang, L., Qi, H., Li, S., & Daxin, L. (2008). Extending BLEU Evaluation Method with Linguistic Weight. 2008 The 9th International Conference for Young Computer Scientists. https://doi.org/10.1109/icycs.2008.362
[103] Yang, M.-Y., Sun, S.-Q., Zhu, J.-G., Li, S., Zhao, T.-J., & Zhu, X.-N. (2011). Improvement of Machine Translation Evaluation by Simple Linguistically Motivated Features. Journal of Computer Science and Technology, 26(1), 57–67. https://doi.org/10.1007/s11390-011-9415-8
[104] Yang, Z. H. (2019). Studies on Translation Testing and Assessment. Foreign Language Teaching and Research Press.
[105] Yao, J.M., Qu, Y.Q., Zhu, Q.M., & Zhang, J. (2006). A Visualization method for machine translation evaluation results. Proceedings (Stymne et al., 2012) of the 20th Pacific Asia Conference on Language, Information and Computation.
[106] Yilmaz, I. (2019). Bibliometric Analysis of Bibliometric Studies on Tourism Published in Turkey. Anais Brasileiros de Estudos Turísticos - ABET, 9(1, 2 e 3). https://doi.org/10.34019/2238-2925.2019.v9.27111
[107] Yunxing, L. (2001). Translation evaluation in a systemic perspective. Perspectives, 9(1), 15–21. https://doi.org/10.1080/0907676x.2001.9961403
[108] Zhu, J., & Wang, H. (2006). The effect of translation quality in MT-based cross-language information retrieval. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL - ACL ’06. https://doi.org/10.3115/1220175.1220250
[109] Zlatnar Moe, M., Mikolic Juznic, T., & Žigon, T. (2015). I know languages, therefore, I can translate? Translation and Interpreting Studies, 10(1), 87–107. Portico. https://doi.org/10.1075/tis.10.1.05moe
Copyright (c) 2024 author(s)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The author(s) warrant that permission to publish the article has not been previously assigned elsewhere.
Author(s) shall retain the copyright of their work and grant the Journal/Publisher right for the first publication with the work simultaneously licensed under:
OA - Creative Commons Attribution License (CC BY 4.0). This license allows for the copying, distribution and transmission of the work, provided the correct attribution of the original creator is stated. Adaptation and remixing are also permitted.
This broad license intends to facilitate free access to, as well as the unrestricted reuse of, original works of all types.
_%E5%89%AF%E6%9C%AC1.png){kind=logo}
