Bibliometric analysis and visualization of translation assessment: Research theme, evolution and hotspots
Abstract
Translation assessment, referring to evaluating various aspects related to translation, is crucial to the improvement of translation competence and quality. This study, taking valid papers on translation assessment published in WoS core collection from 2000 to 2022 as research samples, visualizes and reviews the research theme, research evolutions and emerging hot topics of translation assessment through bibliometric analysis. The number of annual publications shows that despite the general growth, the number of articles on translation assessment fluctuates from year to year. By Cite Space-based analysis of keywords co-occurrence, keyword clustering, time zone and burst detection, it finds that translation assessment mainly covers five themes, namely translation competence, translation quality, machine translation, translation teaching and training, and others. The overall evolution has been a gradual shift from concepts, metrics and evaluations of topics related to translation assessment into a topic of in-depth and interdisciplinary study. The hotspots in recent years include translation competence acquisition, neural machine translation and translation quality estimation. The findings reveal research evolutions and hot spots, propose implications for further translation assessment research and provide references for scholars who are interested in this topic.
References
Adom, D., Adu-Mensah, J., & Dake, D. A. (2020). Test, measurement, and evaluation: Understanding and use of the concepts in education. International Journal of Evaluation and Research in Education (IJERE), 9(1), 109. https://doi.org/10.11591/ijere.v9i1.20457
Airasian, P.W. (1997). Classroom assessment, 3rd ed. McGraw-Hill.
Akulina, K. V., & Tikhonova, E. V. (2021). Application of the translation quality assessment system in the training of linguist-translators. Yazyk i Kul’tura, 56, 129–145. https://doi.org/10.17223/19996195/56/8
Albir, A. H. (2007). Competence-based Curriculum Design for Training Translators. The Interpreter and Translator Trainer, 1(2), 163–195. https://doi.org/10.1080/1750399x.2007.10798757
Albir, P. Group. A. H., Galán-Mañas, A., Kuznik, A., Olalla-Soler, C., Rodríguez-Inés, P., & Romero, L. (2020). Translation competence acquisition. Design and results of the PACTE group’s experimental research. The Interpreter and Translator Trainer, 14(2), 95–233. https://doi.org/10.1080/1750399x.2020.1732601
Alos, J. (2015). Explicating the implicit: an empirical investigation into pragmatic competence in translator training. The Interpreter and Translator Trainer, 9(3), 287–305. https://doi.org/10.1080/1750399x.2015.1100398
al-Qinai, J. (2002). Translation Quality Assessment. Strategies, Parametres and Procedures. Meta, 45(3), 497–519. https://doi.org/10.7202/001878ar
Angelone, E. (2020). The impact of screen recording as a diagnostic process protocol on inter-rater consistency in translation assessment Erik Angelone, Kent State University. Journal of Specialised Translation, 34, 32-50.
Angelone, E. (2021). Broadening the scope of error categories in translation assessment through screen recording. Across Languages and Cultures, 22(2), 143–157. https://doi.org/10.1556/084.2021.00021
Arango-Keeth, F., & Koby, G. S. (2003). Assessing assessment. Beyond the Ivory Tower, 117–134. https://doi.org/10.1075/ata.xii.09ara
Bai, L., Feng, L., & Yan, M. (2018). Constructs and principles of the Chinese English Translation Proficiency Rating Scale. Modem Foreign Language, 41(1), 101-110.
Baniata, L. H., Park, S., & Park, S.-B. (2018). A Multitask-Based Neural Machine Translation Model with Part-of-Speech Tags Integration for Arabic Dialects. Applied Sciences, 8(12), 2502. https://doi.org/10.3390/app8122502
Beeby, A., Castillo, L., Fox, O., Galán Mañas, A., Hurtado Albir, A., Kuznik, A., Massana, G., Neunzig, W., Olalla, Ch., Rodríguez Inés, P., & Romero, L. (2015). Results of PACTE’s Experimental Research on the Acquisition of Translation Competence. Translation Spaces, 4(1), 29–53. Portico. https://doi.org/10.1075/ts.4.1.02bee
Bowker, L. (2002). Towards a Methodology for a Corpus-Based Approach to Translation Evaluation. Meta, 46(2), 345–364. https://doi.org/10.7202/002135ar
Brunette, L. (2000). Towards a Terminology for Translation Quality Assessment. The Translator, 6(2), 169–182. https://doi.org/10.1080/13556509.2000.10799064
Brunette, L. (2000). Towards a Terminology for Translation Quality Assessment. The Translator, 6(2), 169–182. https://doi.org/10.1080/13556509.2000.10799064
Burchardt, A., Lommel, A., Bywood, L., Harris, K., & Popović, M. (2016). Machine translation quality in an audiovisual context. Target, 28(2), 206–221. Portico. https://doi.org/10.1075/target.28.2.03bur
Cao, D. (2014). Teaching and learning legal translation. Semiotica, 2014(201). https://doi.org/10.1515/sem-2014-0022
Carl, M., & Báez, M. C. T. (2019). Machine translation errors and the translation process: A study across different languages. Journal of Specialised Translation, 31, 107-132.
Castilho, S., & O’Brien, S. (2018). Acceptability of machine-translated content: A multi-language evaluation by translators and end-users. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 16. https://doi.org/10.52034/lanstts.v16i0.430
Chakhachiro, R. (2005). Revision for quality. Perspectives, 13(3), 255–238. https://doi.org/10.1080/09076760508668994
Chapelle, C. A., & Brindley, G. (2010). What is language assessment. 247-267.
Chatzikoumi, E. (2019). How to evaluate machine translation: A review of automated and human metrics. Natural Language Engineering, 26(2), 137–161. https://doi.org/10.1017/s1351324919000469
Che, S., Kamphuis, P., Zhang, S., Zhao, X., & Kim, J. H. (2022). A Visualization Analysis of Crisis and Risk Communication Research Using CiteSpace. International Journal of Environmental Research and Public Health, 19(5), 2923. https://doi.org/10.3390/ijerph19052923
Colina, S. (2008). Translation Quality Evaluation. The Translator, 14(1), 97–134. https://doi.org/10.1080/13556509.2008.10799251
Comelles, E., & Atserias, J. (2018). VERTa: a linguistic approach to automatic machine translation evaluation. Language Resources and Evaluation, 53(1), 57–86. https://doi.org/10.1007/s10579-018-9430-2
Cortez-Godínez, J. (2019). Influencia del nivel socioeconómico en la adquisición de la Competencia Traductora. Sendebar, 30(0). https://doi.org/10.30827/sendebar.v30i0.6594
Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L. (2017). Identifying the Machine Translation Error Types with the Greatest Impact on Post-editing Effort. Frontiers in Psychology, 8. https://doi.org/10.3389/fpsyg.2017.01282
De Sutter, G., Cappelle, B., De Clercq, O., Loock, R., & Plevoets, K. (2018). Towards a Corpus-based, Statistical Approach to Translation Quality: Measuring and Visualizing Linguistic Deviance in Student Translations. Linguistica Antverpiensia, New Series – Themes in Translation Studies, 16. https://doi.org/10.52034/lanstts.v16i0.440
Diao, B.C., Adnan, N., Harji, M. B., Ravindran, L. (2022). Evolution and hotspots of peer instruction: a visualized analysis using CiteSpace. Education and Information Technologies. 1-18.
Ding, Z.Y., Zang, H.F., Zhang, Q., Miao, G.M. & Chi, Y.H. Automatic machine translation evaluation based on sentence structure information. Paper presented at the 2009 International Conference on Asian Language Processing.
Du, W., & Saeheaw, T. (2020). Errors in translation: A case study of Chinese into English translation. Language Learning in Higher Education, 10(1), 129–142. https://doi.org/10.1515/cercles-2020-2009
Dugonik, J., Bošković, B., Brest, J., & Sepesy Maučec, M. (2019). Improving Statistical Machine Translation Quality Using Differential Evolution. Informatica, 30(4), 629–645. https://doi.org/10.15388/informatica.2019.222
Dussart, A. (2005). Faux sens, contresens, non-sens… un faux débat ? Meta, 50(1), 107–119. https://doi.org/10.7202/010661ar
Ellegaard, O., & Wallin, J. A. (2015). The bibliometric analysis of scholarly production: How great is the impact? Scientometrics, 105(3), 1809–1831. https://doi.org/10.1007/s11192-015-1645-z
Erten, C., Harding, P. J., Kobourov, S. G., Wampler, K., & Yee, G. (2004). Exploring the computing literature using temporal graph visualization SPIE Proceedings. https://doi.org/10.1117/12.539245
Finch, A., Yasuda, K., Okuma, H., Sumita, E., Nakamura, S. (2011). A Bayesian Model of Transliteration and Its Human Evaluation When Integrated into a Machine Translation System. IEICE Transactions on Information and Systems, E94-D(10), 1889–1900. https://doi.org/10.1587/transinf.e94.d.1889
Fomicheva, M., & Specia, L. (2019). Taking MT Evaluation Metrics to Extremes: Beyond Correlation with Human Judgments. Computational Linguistics, 45(3), 515–558. https://doi.org/10.1162/coli_a_00356
Gambier, Y., & van Doorslaer, L. (Eds.). (2007). The Metalanguage of Translation. Target, 19(2). Portico. https://doi.org/10.1075/target.19.2
Garcia, I. (2010). Is machine translation ready yet? Target, 22(1), 7–21. Portico. https://doi.org/10.1075/target.22.1.02gar
Gile, D. (2000). The History of Research into Conference Interpreting. Target, 12(2), 297–321. Portico. https://doi.org/10.1075/target.12.2.07gil
Göpferich, S. (2013). Translation competence. Target, 25(1), 61–76. Portico. https://doi.org/10.1075/target.25.1.06goe
Guo, Y.-M., Huang, Z.-L., Guo, J., Li, H., Guo, X.-R., & Nkeli, M. J. (2019). Bibliometric Analysis on Smart Cities Research. Sustainability, 11(13), 3606. https://doi.org/10.3390/su11133606
Guzmán, F., Joty, S., Màrquez, L., & Nakov, P. (2017). Machine translation evaluation with neural networks. Computer Speech & Language, 45, 180–200. https://doi.org/10.1016/j.csl.2016.12.005
Hague, D. R. (2015). Empathic accuracy and translator training: helping students imagine other minds. The Interpreter and Translator Trainer, 9(1), 20–29. https://doi.org/10.1080/1750399x.2015.1010359
Hague, D., Melby, A., & Zheng, W. (2011). Surveying Translation Quality Assessment. The Interpreter and Translator Trainer, 5(2), 243–267. https://doi.org/10.1080/13556509.2011.10798820
Hajmohammadi, A. (2005). TRANSLATION EVALUATION IN A NEWS AGENCY. Perspectives, 13(3), 215–224. https://doi.org/10.1080/09076760508668993
Han, C. (2020). Translation quality assessment: a critical methodological review. The Translator, 26(3), 257–273. https://doi.org/10.1080/13556509.2020.1834751
Han, C., Hu, B., Fan, Q., Duan, J., & Li, X. (2022). Using computerised comparative judgement to assess translation. Across Languages and Cultures, 23(1), 56–74. https://doi.org/10.1556/084.2022.00001
Han, Y., & Meng, S. (2022). Machine English Translation Evaluation System Based on BP Neural Network Algorithm. Computational Intelligence and Neuroscience, 2022, 1–10. https://doi.org/10.1155/2022/4974579
Hassan, H., Hearne, M., Way, A., & Sima’an, K. (2006). Syntactic phrase-based statistical machine translation. Paper presented at the 2006 IEEE Spoken Language Technology Workshop. https://doi.org/10.1109/slt.2006.326799
House, J. (2002). Translation Quality Assessment: Linguistic Description versus Social Evaluation. Meta, 46(2), 243–257. https://doi.org/10.7202/003141ar
Huang, Q., & Xin, X. (2020). A bibliometric analysis of translation criticism studies and its implications. Perspectives, 28(5), 737–755. https://doi.org/10.1080/0907676x.2020.1740750
Janmaijaya, M., Shukla, A., Abraham, A., & Muhuri, P. (2018). A Scientometric Study of Neurocomputing Publications (1992–2018): An Aerial Overview of Intrinsic Structure. Publications, 6(3), 32. https://doi.org/10.3390/publications6030032
Jiang, C. (2010). Quality assessment for the translation of museum texts: application of a systemic functional model. Perspectives, 18(2), 109–126. https://doi.org/10.1080/09076761003678734
Károly, A. (2014). Translation in foreign language teaching: A case study from a functional perspective. Linguistics and Education, 25, 90–107. https://doi.org/10.1016/j.linged.2013.09.009
Karoubi, B. (2016). Translation quality assessment demystified. Babel, 62(2), 253–277. Portico. https://doi.org/10.1075/babel.62.2.05kar
Kasperavičienė, R., & Horbačauskienė, J. (2020). Self-revision and other-revision as part of translation competence in translator training. Journal of Language and Cultural Education, 8(1), 117–133. https://doi.org/10.2478/jolace-2020-0007
Ke, X.H. (2009). An Automatic Translation Evaluation System Based on Semantic Similarities and Fuzzy Neartude. Paper presented at the 2009 International Conference on Environmental Science and Information Application Technology.
Kiraly, D. (2000). A Social Constructivist Approach to Translator Education: Empowerment from Theory to Practice. St Jerome Publishing.
Kit, C., & Wong, T. M. (2008). Comparative evaluation of online machine translation systems with legal texts. Law Library Journal, 100(2), 299-321.
Korol, T. (2020). Translation assessment: Is there anything to be tested objectively? Advanced Education, 15, 32-40. https://doi.org/10.20535/2410-8286.200720
Korol, T. (2021). Digital teacher feedback as a translation assessment tool. Journal of Teaching English for Specific and Academic Purposes, 575. https://doi.org/10.22190/jtesap2104575k
Kudo, T. (2018). Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/p18-1007
Lee, Y. O., & Ronowicz, E. (2014). The development of an error typology to assess translation from English into Korean in class. Babel, 60(1), 35–51. Portico. https://doi.org/10.1075/babel.60.1.03lee
Li, D. (2006). Making Translation Testing More Teaching-oriented: A Case Study of Translation Testing in China. Meta, 51(1), 72–88. https://doi.org/10.7202/012994ar
Liu, L., Fujita, A., Utiyama, M., Finch, A., & Sumita, E. (2017). Translation Quality Estimation Using Only Bilingual Corpora. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 25(9), 1762–1772. https://doi.org/10.1109/taslp.2017.2716195
Lommel, A., Uszkoreit, H., & Burchardt, A. (2014). Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics. Tradumàtica Tecnologies de La Traducció, 12, 455–463. https://doi.org/10.5565/rev/tradumatica.77
Maier, C. (2000). Introduction. The Translator, 6(2), 137–148. https://doi.org/10.1080/13556509.2000.10799062
Maier, C. (2014). Evaluation and Translation. Routledge. https://doi.org/10.4324/9781315759999
Martínez Melis, N., & Hurtado Albir, A. (2002). Assessment In Translation Studies: Research Needs. Meta, 46(2), 272–287. https://doi.org/10.7202/003624ar
McAlester, G. (1999). “The Source Text in Translation Assessment”. In Words, Words, Words. The Translator and the Language Learner. In: Anderman G, Rogers, M. (editors). pp. 169-178. Clevedon, Philadelphia and Adelaide: Multilingual Matters.
Mei, H., & Chen, H. (2022). Assessing Students’ Translation Competence: Integrating China’s Standards of English With Cognitive Diagnostic Assessment Approaches. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.872025
Mellinger, C. D. (2018). Re-thinking translation quality. Target, 310–331. https://doi.org/10.1075/target.16104.mel
Moorkens, J. (2018). What to expect from Neural Machine Translation: a practical in-class translation evaluation exercise. The Interpreter and Translator Trainer, 12(4), 375–387. https://doi.org/10.1080/1750399x.2018.1501639
Munday, J. (2016). Introducing Translation Studies. Routledge. https://doi.org/10.4324/9781315691862
Nida, E., & Taber, C. (2003). The Theory and Practice of Translation. https://doi.org/10.1163/9789004496330
Nikolaeva, S., & Korol, T. (2021). Prospective Philologists’ translation assessment triangulation: screen video recording and think aloud protocol combination. Advanced Education, 8(18), 30–41. LOCKSS. https://doi.org/10.20535/2410-8286.228550
Olawumi, T. O., & Chan, D. W. M. (2018). A scientometric review of global research on sustainability and sustainable development. Journal of Cleaner Production, 183, 231–250. https://doi.org/10.1016/j.jclepro.2018.02.162
Orozco, M., & Hurtado Albir, A. (2004). Measuring Translation Competence Acquisition. Meta, 47(3), 375–402. https://doi.org/10.7202/008022ar
PACET (2019). Evolution of the Efficacy of the Translation Process in Translation Competence Acquisition. Meta: Journal Des Traducteurs, 64(1), 242. https://doi.org/10.7202/1065336ar
PACTE (2005). Investigating translation competence: conceptual and methodological issues. Volet Traduction, 50(2), 609–619. https://doi.org/10.7202/011004ar
Palumbo, G. (2009). Key terms in translation studies: Bloomsbury Publishing.
Pan, W., Jian, L., & Liu, T. (2019). Grey system theory trends from 1991 to 2018: a bibliometric analysis and visualization. Scientometrics, 121(3), 1407–1434. https://doi.org/10.1007/s11192-019-03256-z
Pollack, J., & Adler, D. (2015). Emergent trends and passing fads in project management research: A scientometric analysis of changes in the field. International Journal of Project Management, 33(1), 236–248. https://doi.org/10.1016/j.ijproman.2014.04.011
Popescu, T. (2013). A Corpus-based Approach to Translation Error Analysis. a Case-study of Romanian EFL Learners. Procedia - Social and Behavioral Sciences, 83, 242–247. https://doi.org/10.1016/j.sbspro.2013.06.048
Pranckutė, R. (2021). Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World. Publications, 9(1), 12. https://doi.org/10.3390/publications9010012
Pym, A. (2004). Redefining Translation Competence in an Electronic Age. In Defence of a Minimalist Approach. Meta, 48(4), 481–497. https://doi.org/10.7202/008533ar
Pym, A. (2013). Research skills in translation studies: What we need training in. Across Languages and Cultures, 14(1), 1–14. https://doi.org/10.1556/acr.14.2013.1.1
Sadat, F., & Habash, N. (2006). Combination of Arabic preprocessing schemes for statistical machine translation. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL - ACL ’06. https://doi.org/10.3115/1220175.1220176
Schwenk, H., & Esteve, Y. (2008). Data selection and smoothing in an open-source system for the 2008 NIST machine translation evaluation. Interspeech 2008. https://doi.org/10.21437/interspeech.2008-676
Şerban, A., & Mureşan, V. (2019). Register and Translation Competence Acquisition. Investigating Student Attitudes. Studia Universitatis Babeș-Bolyai Philologia, 64(4), 213–230. https://doi.org/10.24193/subbphilo.2019.4.13
Si, Y.H., Shi, J.G., Chen, J.D., Wu, G.D., & Wang, H.M., (2019). Reviewing sustainable development research in China from the perspective of China Population, Resources and Environment. China population, resources and environment, 29(7), 166-176.
Su, W. (2021). Understanding rubric use in peer assessment of translation. Perspectives, 30(1), 71–85. https://doi.org/10.1080/0907676x.2020.1862260
Tingting, L., & Mengyu, X. (2019). Analysis and evaluation on the quality of news text machine translation based on neural network. Multimedia Tools and Applications, 79(23–24), 17015–17026. https://doi.org/10.1007/s11042-019-7532-5
Tymoczko, M. (2006). Trajectories of Research in Translation Studies. Séances Plénières, 50(4), 1082–1097. https://doi.org/10.7202/012062ar
Waddington, C. (2002). Different Methods of Evaluating Student Translations: The Question of Validity. Meta, 46(2), 311–325. https://doi.org/10.7202/004583ar
Wang, H., Wu, H., He, Z., Huang, L., & Church, K. W. (2022). Progress in Machine Translation. Engineering, 18, 143–153. https://doi.org/10.1016/j.eng.2021.03.023
Wang, L., & Wang, X. (2020). How to evaluate literary translations in the classroom context: through error analysis or scale-based method? Current Trends in Translation Teaching and Learning E, 7, 276-313.
Williams, M. (2002). The Application of Argumentation Theory to Translation Quality Assessment. Meta, 46(2), 326–344. https://doi.org/10.7202/004605ar
Xiao, W.Q. (2012). Study on the Undergraduate Translation Major Testing. People’s Publishing House.
Yang, M., Zhu, J., Li, J., Wang, L., Qi, H., Li, S., & Daxin, L. (2008). Extending BLEU Evaluation Method with Linguistic Weight. 2008 The 9th International Conference for Young Computer Scientists. https://doi.org/10.1109/icycs.2008.362
Yang, M.-Y., Sun, S.-Q., Zhu, J.-G., Li, S., Zhao, T.-J., & Zhu, X.-N. (2011). Improvement of Machine Translation Evaluation by Simple Linguistically Motivated Features. Journal of Computer Science and Technology, 26(1), 57–67. https://doi.org/10.1007/s11390-011-9415-8
Yang, Z. H. (2019). Studies on Translation Testing and Assessment. Foreign Language Teaching and Research Press.
Yao, J.M., Qu, Y.Q., Zhu, Q.M., & Zhang, J. (2006). A Visualization method for machine translation evaluation results. Proceedings (Stymne et al., 2012) of the 20th Pacific Asia Conference on Language, Information and Computation.
Yilmaz, I. (2019). Bibliometric Analysis of Bibliometric Studies on Tourism Published in Turkey. Anais Brasileiros de Estudos Turísticos - ABET, 9(1, 2 e 3). https://doi.org/10.34019/2238-2925.2019.v9.27111
Yunxing, L. (2001). Translation evaluation in a systemic perspective. Perspectives, 9(1), 15–21. https://doi.org/10.1080/0907676x.2001.9961403
Zhu, J., & Wang, H. (2006). The effect of translation quality in MT-based cross-language information retrieval. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL - ACL ’06. https://doi.org/10.3115/1220175.1220250
Zlatnar Moe, M., Mikolic Juznic, T., & Žigon, T. (2015). I know languages, therefore, I can translate? Translation and Interpreting Studies, 10(1), 87–107. Portico. https://doi.org/10.1075/tis.10.1.05moe
Copyright (c) 2024 author(s)
This work is licensed under a Creative Commons Attribution 4.0 International License.
The author(s) warrant that permission to publish the article has not been previously assigned elsewhere.
Author(s) shall retain the copyright of their work and grant the Journal/Publisher right for the first publication with the work simultaneously licensed under:
OA - Creative Commons Attribution License (CC BY 4.0). This license allows for the copying, distribution and transmission of the work, provided the correct attribution of the original creator is stated. Adaptation and remixing are also permitted.
This broad license intends to facilitate free access to, as well as the unrestricted reuse of, original works of all types.
_%E5%89%AF%E6%9C%AC1.png){kind=logo}
