IIIT Hyderabad Publications |
|||||||||
|
Evaluating Dialogs based on Grice’s MaximsAuthors: Prathyusha Jwalapuram Conference: Recent Advances in Natural Language Processing (RANLP-2017 2017) Location Varna, Bulgaria Date: 2017-09-02 Report no: IIIT/TR/2017/85 AbstractThere is no agreed upon standard for the evaluation of conversational dialog systems, which are well-known to be hard to evaluate due to the difficulty in pinning down metrics that will correspond to human judgements and the subjective nature of human judgment itself. We explored the possibility of using Grice’s Maxims to evaluate effective communication in conversation. We collected some system generated dialogs from popular conversational chatbots across the spectrum and conducted a survey to see how the human judgements based on Gricean maxims correlate, and if such human judgments can be used as an effective evaluation metric for conversational dialog. Full paper: pdf Centre for Language Technologies Research Centre |
||||||||
Copyright © 2009 - IIIT Hyderabad. All Rights Reserved. |