[UCL WI Talks]: Reliable, Adaptable, and Attributable LMs with Retrieval

Abstract:
Parametric language models (LMs), trained on vast web data, offer impressive flexibility yet pose challenges like hallucinations, and expensive costs for adapting to new data distributions. In this talk, I'll discuss why retrieval-augmented LMs represent the next LM generation and present our recent work in advancing them for wider adoption. First, I'll explore the promises and limits of retrieval-augmented LMs. By integrating large-scale datastores during inference, they become more reliable, adaptable, and attributable, addressing several inherent issues found in parametric LMs. Despite their potential, their adoption remains limited. I'll outline technical hurdles and a roadmap for progress. Next, I'll detail our recent efforts to enhance both retriever and LM components through novel training and inference techniques, to address those limitations for more versatile and efficient retrieval-augmented LMs. Lastly, I'll touch upon exciting future research avenues, including refined retrieval systems, improving architectures with enhanced retriever-LM interactions, and exploring applications in safety-critical domains.

Bio:
Akari Asai is a Ph.D. student in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, advised by Prof. Hannaneh Hajishirzi. Her research is centered around natural language processing and machine learning, with a recent focus on advancing retrieval-augmented language models (LMs) and representation and retrieval systems for these models. She co-taught the first tutorial on retrieval-augmented LMs. Her work has been recognized with multiple awards, including fellowships such as the IBM Fellowship and the Nakajima Foundation Fellowship, as well as paper awards at conferences like ACL 2023 and the NeurIPS Workshop on Instruction Tuning. She was also selected as one of the EECS Rising Stars in 2022.