Synergy with Translation Artifacts for Training and Inference in Multilingual Tasks

October 18, 2022 · Entered Twilight · 🏛 Conference on Empirical Methods in Natural Language Processing

Repo contents: .gitignore, README.md, data, data_loader, evaluation.ipynb, exps, finetuning_baseline.py, finetuning_parameters.py, future, requirements.txt, translate.py, utils

Authors Jaehoon Oh, Jongwoo Ko, Se-Young Yun arXiv ID 2210.09588 Category cs.CL: Computation & Language Citations 9 Venue Conference on Empirical Methods in Natural Language Processing Repository https://github.com/jongwooko/MUSC ⭐ 2 Last Checked 1 month ago

Abstract

Translation has played a crucial role in improving the performance on multilingual tasks: (1) to generate the target language data from the source language data for training and (2) to generate the source language data from the target language data for inference. However, prior works have not considered the use of both translations simultaneously. This paper shows that combining them can synergize the results on various multilingual sentence classification tasks. We empirically find that translation artifacts stylized by translators are the main factor of the performance gain. Based on this analysis, we adopt two training methods, SupCon and MixUp, considering translation artifacts. Furthermore, we propose a cross-lingual fine-tuning algorithm called MUSC, which uses SupCon and MixUp jointly and improves the performance. Our code is available at https://github.com/jongwooko/MUSC.