Hubei University of Science and Technology, Xianning, Hubei, China.
Nanjing University of Information Science and Technology, Nanjing, China.
Comput Intell Neurosci. 2022 Aug 8;2022:8490760. doi: 10.1155/2022/8490760. eCollection 2022.
Legal judgment prediction is the most typical application of artificial intelligence technology, especially natural language processing methods, in the judicial field. In a practical environment, the performance of algorithms is often restricted by the computing resource conditions due to the uneven computing performance of the devices. Reducing the computational resource consumption of the model and improving the inference speed can effectively reduce the deployment difficulty of the legal judgment prediction model. To improve the prediction accuracy, enhance the model inference speed, and reduce the model memory consumption, we propose a BERT knowledge distillation-based legal decision prediction model, called KD-BERT. To reduce the resource consumption in the model inference process, we use the BERT pretraining model with lower memory requirements to be the encoder. Then, the knowledge distillation strategy transfers the knowledge to the student model of the shallow transformer structure. Experiment results show that the proposed KD-BERT has the highest F1-score compared with traditional BERT models. Its inference speed is also much faster than the other BERT models.
法律判决预测是人工智能技术，尤其是自然语言处理方法，在司法领域中最典型的应用。在实际环境中，由于设备的计算性能不均衡，算法的性能往往受到计算资源条件的限制。减少模型的计算资源消耗和提高推理速度可以有效地降低法律判决预测模型的部署难度。为了提高预测精度、增强模型推理速度和减少模型内存消耗，我们提出了一种基于 BERT 知识蒸馏的法律决策预测模型，称为 KD-BERT。为了减少模型推理过程中的资源消耗，我们使用内存需求较低的 BERT 预训练模型作为编码器。然后，知识蒸馏策略将知识转移到浅层转换器结构的学生模型中。实验结果表明，与传统的 BERT 模型相比，所提出的 KD-BERT 具有最高的 F1 分数。它的推理速度也比其他 BERT 模型快得多。
