Improving Retriever Performance in Handling Ambiguous Queries

Abstract

In the realm of open-domain question answering, a common challenge arises from the inherent ambiguity of user queries. Conventional question-answering systems that offer a single answer often falter when confronted with ambiguity, as questions may be subject to multiple interpretations and yield various distinct answers. This paper addresses this challenge through the lens of multi-answer retrieval, a task focused on retrieving passages capable of capturing the diverse array of answers to a single question. Our approach introduces a re-ranking methodology that leverages Determinantal point processes, employing BERT embeddings as kernels. This technique takes a holistic view by jointly considering both query-passage relevance and passage-passage correlation. The goal is to retrieve passages that not only align with the user’s query but also encompass a diversity of information. Empirical results underscore the effectiveness of our re-ranking approach, demonstrating its superiority over state-of-the-art methods, particularly when evaluated on the AmbigQA dataset. In parallel, the field of question-answering, powered by reader models or answer generation, has seen significant advancements. Neural sequence-to-sequence models, especially those incorporating attention mechanisms, have played a vital role in this progress. However, these models encounter a limitation as they predominantly rely on pointer generators that exclusively target words from the source passage, even though their goal is to provide answers that involve both the question and the source text. To address this constraint, we introduce an innovative query pointer module within a comprehensive multipointer generator framework. This module facilitates the generation of answers that combine question and passage information in diverse ways, resulting in contextually relevant responses that effectively address the nuances of user queries. Our model demonstrates a significant improvement of 3.5 points in Rouge-L scores when compared to the state-of-the-art model on existing datasets. These advancements collectively signify substantial progress in both multi-answer retrieval and question-answering research, offering the promise of enhanced capabilities in information retrieval, natural language understanding, and content summarization.