Loading...
Please wait, while we are loading the content...
Similar Documents
Semi-supervised Neural Machine Translation with Consistency Regularization for Low-Resource Languages
| Content Provider | arXiv |
|---|---|
| Author | Pham, Viet H. Pham, Thang M. Nguyen, Giang Nguyen, Long Dinh, Dien |
| Date of Submission | 2023-04-02 |
| Abstract | The advent of deep learning has led to a significant gain in machine translation. However, most of the studies required a large parallel dataset which is scarce and expensive to construct and even unavailable for some languages. This paper presents a simple yet effective method to tackle this problem for low-resource languages by augmenting high-quality sentence pairs and training NMT models in a semi-supervised manner. Specifically, our approach combines the cross-entropy loss for supervised learning with KL Divergence for unsupervised fashion given pseudo and augmented target sentences derived from the model. We also introduce a SentenceBERT-based filter to enhance the quality of augmenting data by retaining semantically similar sentence pairs. Experimental results show that our approach significantly improves NMT baselines, especially on low-resource datasets with 0.46--2.03 BLEU scores. We also demonstrate that using unsupervised training for augmented data is more efficient than reusing the ground-truth target sentences for supervised learning. |
| Related Links | https://arxiv.org/pdf/2304.00557.pdf |
| arXiv | 2304.00557 |
| Language | English |
| Access Restriction | Open |
| Subject Keyword | Computer Science - Computation and Language Computer Science - Machine Learning Computer Science |
| Content Type | Text |
| Resource Type | Article |
| Subject | Computer Science |
