This paper addresses the misalignment between traditional IR evaluation metrics and the requirements of modern Retrieval-Augmented Generation (RAG) systems. It proposes a novel annotation schema and the UDCG metric to better evaluate retrieval quality for LLM consumers.
