Models Don’t Get Partial Credit: An Analysis of Contradictory Evidence on Multi-hop Question Answering Chains of Reasoning

Abstract

While question answering (QA) models perform well in open-book settings, these models can create false chains of reasoning in the presence of adversarially contradictory evidence. In doing so, such models engage in a form of confirmation bias that may be misleading or even harmful to an information-seeking agent. We evaluate how different forms of adversarial data affect multi-hop QA model reasoning, using a RoBERTa model pre-trained on HotpotQA, and find that multi-hop QA systems are affected by contradictory evidence 44.75% of the time. We introduce contradictory information in the evidence sources and evaluate multi-hop QA reasoning by manually validating our approach to show the impact of including contradictory evidence in multi-hop QA using the HotpotQA dataset.