Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh P.O. Box 11671, Saudi Arabia.
Department of Information Systems, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, Riyadh P.O. Box 11671, Saudi Arabia.
Sensors (Basel). 2021 Feb 21;21(4):1493. doi: 10.3390/s21041493.
In this paper, we introduce new concepts in the machine translation paradigm. We treat the corpus as a database of frequent word sets. A translation request triggers association rules joining phrases present in the source language, and phrases present in the target language. It has to be noted that a sequential scan of the corpus for such phrases will increase the response time in an unexpected manner. We introduce the pre-processing of the bilingual corpus through proposing a data structure called Corpus-Trie (CT) that renders a bilingual parallel corpus in a compact data structure representing frequent data items sets. We also present algorithms which utilize the CT to respond to translation requests and explore novel techniques in exhaustive experiments. Experiments were performed on specific language pairs, although the proposed method is not restricted to any specific language. Moreover, the proposed Corpus-Trie can be extended from bilingual corpora to accommodate multi-language corpora. Experiments indicated that the response time of a translation request is logarithmic to the count of unrepeated phrases in the original bilingual corpus (and thus, the Corpus-Trie size). In practical situations, 5-20% of the log of the number of the nodes have to be visited. The experimental results indicate that the BLEU score for the proposed CT system increases with the size of the number of phrases in the CT, for both English-Arabic and English-French translations. The proposed CT system was demonstrated to be better than both Omega-T and Apertium in quality of translation from a corpus size exceeding 1,600,000 phrases for English-Arabic translation, and 300,000 phrases for English-French translation.
在本文中，我们引入了机器翻译范式中的新概念。我们将语料库视为频繁词集的数据库。翻译请求会触发将源语言中出现的短语和目标语言中出现的短语联系起来的关联规则。需要注意的是，对语料库进行这种短语的顺序扫描会以意想不到的方式增加响应时间。我们通过提出一种称为语料库 trie (CT) 的数据结构来预处理双语语料库，该数据结构将双语平行语料库表示为一个紧凑的数据结构，代表频繁的数据项集。我们还提出了利用 CT 来响应翻译请求并在详尽的实验中探索新的技术的算法。实验是在特定的语言对上进行的，尽管所提出的方法不限于任何特定的语言。此外，所提出的语料库 trie 可以从双语语料库扩展到多语言语料库。实验表明，翻译请求的响应时间与原始双语语料库中未重复短语的数量的对数（因此，语料库 trie 的大小）成对数关系。在实际情况下，必须访问原始双语语料库中未重复短语数量的日志的 5-20%。实验结果表明，对于英语-阿拉伯语和英语-法语翻译，所提出的 CT 系统的 BLEU 分数随着 CT 中短语数量的增加而增加。对于英语-阿拉伯语翻译超过 160 万短语和英语-法语翻译超过 30 万短语的语料库大小，所提出的 CT 系统在翻译质量上优于 Omega-T 和 Apertium。
