This is a common problem for models that have been trained on small amounts of data, and affects many low-resource languages. In addition to general quality improvements, the new models show increased robustness to machine translation hallucination, a phenomenon in which models produce strange “translations” when given nonsense input. For this reason, we performed human side-by-side evaluations on all new models, which confirmed the gains in BLEU. For instance, several works have demonstrated how the BLEU score can be biased by translationese effects on the source side or target side, a phenomenon where translated text can sound awkward, containing attributes (like word order) from the source language. This improvement is comparable to the gain observed four years ago when transitioning from phrase-based translation to NMT.Īlthough BLEU score is a well-known approximate measure, it is known to have various pitfalls for systems that are already high-quality. With these latest updates, we see an average BLEU gain of +5 points over the previous GNMT models, with the 50 lowest-resource languages seeing an average gain of +7 BLEU. The improvements since the implementation of the new techniques over the last year are highlighted at the end of the animation.Īdvances for Both High- and Low-Resource LanguagesĪ popular metric for automatic quality evaluation of machine translation systems is the BLEU score, which is based on the similarity between a system’s translation and reference translations that were generated by people. The quality improvements, which averaged +5 BLEU score over all 100+ languages, are visualized below.īLEU score of Google Translate models since shortly after its inception in 2006. These techniques span improvements to model architecture and training, improved treatment of noise in datasets, increased multilingual transfer learning through M4 modeling, and use of monolingual data. In this post, we share some recent progress we have made in translation quality for supported languages, especially for those that are low-resource, by synthesizing and expanding a variety of recent advances, and demonstrate how they can be applied at scale to noisy, web-mined data. Many techniques have demonstrated significant gains for low-resource languages in controlled research settings (e.g., the WMT Evaluation Campaign), however these results on smaller, publicly available datasets may not easily transition to large, web-crawled datasets. And while the research community has developed techniques that are successful for high-resource languages like Spanish and German, for which there exist copious amounts of training data, performance on low-resource languages, like Yoruba or Malayalam, still leaves much to be desired. Nevertheless, state-of-the-art systems lag significantly behind human performance in all but the most specific translation tasks. Therefore one might have to be a little flexible or creative with grammar to make it work.Posted by Isaac Caswell and Bowen Liang, Software Engineers, Google ResearchĪdvances in machine learning (ML) have driven improvements to automated translation, including the GNMT neural translation model introduced in Translate in 2016, that have enabled great improvements to the quality of translation for over 100 languages. The translation is a little forced the restriction of making the words go nicely in a square is too much for fluent language unless one has extraordinary luck. Having specifically a plough seems to come more from sator (what tools could one use in such work?) than rotas. Interpreting a sower as a farmer is reasonable, but taking wheels to stand for a plough seems to be a farther fetch. It is not more specific than "there is a work-related reason that the sower Arepo holds wheels", but that feels clumsy as a translation. The translation "as a form of work" makes sense, too. Starting with the other words, Sator Arepo tenet rotas means "the sower Arepo has/holds wheels".Īdding an ablative to describe circumstances, we get something like "the sower Arepo has/holds wheels for (the purpose of) work". I think the intention is that opera is a singular ablative of the feminine noun opera, not a form of opus.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |