Zheng Jiangbin, Zhao Zheng, Chen Min, Chen Jing, Wu Chong, Chen Yidong, Shi Xiaodong, Tong Yiqi
Department of Artificial Intelligence, School of Informatics, Xiamen University, Xiamen 361005, China.
China Mobile (Suzhou) Software Technology Co., LTD, Suzhou 215000, China.
Comput Intell Neurosci. 2020 Oct 23;2020:8816125. doi: 10.1155/2020/8816125. eCollection 2020.
(SLT) is an important application to bridge the communication gap between deaf and hearing people. In recent years, the research on the SLT based on neural translation frameworks has attracted wide attention. Despite the progress, current SLT research is still in the initial stage. In fact, current systems perform poorly in processing long sign sentences, which often involve long-distance dependencies and require large resource consumption. To tackle this problem, we propose two explainable adaptations to the traditional neural SLT models using optimized tokenization-related modules. First, we introduce a (FSDC) algorithm for detecting and reducing the redundant similar frames, which effectively shortens the long sign sentences without losing information. Then, we replace the traditional encoder in a (NMT) module with an improved architecture, which incorporates a (T-Conv) unit and a (DH-BiGRU) unit sequentially. The improved component takes the temporal tokenization information into consideration to extract deeper information with reasonable resource consumption. Our experiments on the dataset show that the proposed model outperforms the state-of-the-art baseline up to about 1.5+ BLEU-4 score gains.
手语翻译（SLT）是弥合聋人和听力正常人之间沟通鸿沟的一项重要应用。近年来，基于神经翻译框架的手语翻译研究受到了广泛关注。尽管取得了进展，但当前的手语翻译研究仍处于初始阶段。事实上，当前系统在处理长手语句子时表现不佳，长手语句子往往涉及长距离依赖关系且需要大量资源消耗。为解决这一问题，我们提出了两种使用优化的分词相关模块对传统神经手语翻译模型进行的可解释性改编。首先，我们引入一种用于检测和减少冗余相似帧的快速相似帧检测与压缩（FSDC）算法，该算法能有效缩短长手语句子且不丢失信息。然后，我们在神经机器翻译（NMT）模块中用一种改进架构替换传统编码器，该改进架构依次包含一个时间卷积（T-Conv）单元和一个双向门控循环单元（DH-BiGRU）。改进后的组件考虑了时间分词信息，以合理的资源消耗提取更深入的信息。我们在[具体数据集名称]数据集上的实验表明，所提出的模型比最先进的基线模型性能提升高达约1.5个以上的BLEU-4分数增益。
