Answering questions in narratives about why events happened often requires commonsense knowledge external to the text. What aspects of this knowledge are available in large language models? What aspects can be made accessible via external commonsense resources? We study these questions in the context of answering questions in the TELLMEWHY dataset using COMET as a source of relevant commonsense relations. We analyze the effects of model size (T5 variants and GPT-3) along with methods of injecting knowledge (COMET) into these models. Results show that the largest models, as expected, yield substantial improvements over base models and injecting external knowledge helps models of all sizes. We also find that the format in which knowledge is provided is critical, and that smaller models benefit more from larger amounts of knowledge. Finally, we develop an ontology of knowledge types and analyze the relative coverage of the models across these categories.

PDF

In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, December 2022.

@inproceedings{lal:emnlp22, title={Using Commonsense Knowledge to Answer Why-Questions}, author={Yash Kumar Lal and Niket Tandon and Tanvi Aggarwal and Horace Liu and Nathanael Chambers and Raymond Mooney and Niranjan Balasubramanian}, booktitle={Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing}, month={December}, url="http://www.cs.utexas.edu/users/ai-lab?lal:emnlp22", year={2022} }

Video