气有浩然 学无止境

基于单向Transformer和孪生网络的多轮任务型对话技术研究

基于单向 Transformer 和孪生网络的多轮任务型对话技术研究

作者:王 涛 刘超辉 郑青青 黄嘉曦

摘要:为了解决循环神经网络和 Transformer 在多轮对话系统的建模上依赖于大量的样本数据,且回复 的准确率过低的问题,该研究提出了一种针对任务型对话系统的新的建模方式。首先,引入预训练模型对 话过程进行深度编码,其次对 Transformer 模型进行精简,仅保留编码器部分的单向 Transformer,然后 将应答部分抽象成不同的指令,采用孪生网络对指令进行基于相似度的排序,最终选择相似度最高的指令 生成应答。在 MultiWOZ 数据集上的实验结果表明,与 LSTM 和当前最先进的基于 Transformer 的模型 相比,该研究提出的方法预测速度更快,小数据集上的表现也更加优秀,在大数据集上也能取得和最先进 模型相当的效果。

MULTI-TURN TASK-ORIENTED DIALOGUE SYSTEM BASED ON UNIDIRECTIONAL TRANSFORMER AND SIAMESE NETWORK

Author:Wang Tao, Liu Chaohui, Zheng Qingqing, Huang Jiaxi

Abstract:In order to solve the problem that the recurrent neural network and Transformer rely on a large datasets in the modeling of the multi-turn dialogue system, and the accuracy of the reply is too low. this work presented a new modeling method, especially for the Task-Oriented dialogue system. We used the pre-training models and tools to encode the dialog contents deeply, then simplified the Transformer model, only kept the encoder of it. After that, we abstract the response to different commands, and sorted them by Siamese Network. At last, we choose the best result to generate response. The experimental results on the MultiWOZ datasets show that, compared to LSTM and State-of-the-art model based on Transformer, the method used in this study has faster prediction speed, performs better on small datasets, and achieves comparable results on large datasets.

论文下载
知网链接

⬆️