Designing Conversational Search for Libraries

Retrieval Augmented Generation through Open Source Large Language Models

Abstract

Large language models (LLMs) from the commercial domain like BERT and GPT have made machine learning technologies accessible to everyone. On the other hand, the open-source LLMs like Llama, Mistral, and Orca are equally effective and are now widely available. Librarians and information professionals around the world are exploring how to use these models to improve library systems, particularly in the area of searching and finding information, and in building question-answer based search systems. This research study aims to use open-source large language models to develop a conversational search system that can answer questions in natural language on the basis of a given set of documents. The system is based on a Retrieval Augmented Generation (RAG) pipeline, which helps to overcome two major issues with large language models: providing false or imaginary information (hallucination) and giving outdated or unrelated answers. Through two case studies, this research demonstrates that using a RAG-based approach can effectively address these issues and provide more accurate and relevant results. The study proves that an open-source RAG framework can be used to incorporate large language models into library search systems. This integration allows users to receive direct answers to their questions, rather than just a list of potentially relevant documents. In the coming future, the conversational search system can be designed to work in Indian languages, allowing users to ask questions and receive answers in their preferred language.