This article introduces the pyramid search approach using Agentic Knowledge Distillation to address the limitations of traditional RAG strategies in document ingestion.
The pyramid structure allows for multi-level retrieval, including atomic insights, concepts, abstracts, and recollections. This structure mimics a knowledge graph but uses natural language, making it more efficient for LLMs to interact with.
Knowledge Distillation Process:
- Conversion to Markdown: Documents are converted to Markdown for better token efficiency and processing.
- Atomic Insights Extraction: Each page is processed using a two-page sliding window to generate a list of insights in simple sentences.
- Concept Distillation: Higher-level concepts are identified from the insights to reduce noise and preserve essential information.
- Abstract Creation: An LLM writes a comprehensive abstract for each document, capturing dense information efficiently.
- Recollections/Memories: Critical information useful across all tasks is stored at the top of the pyramid.
A collection of lightweight AI-powered tools built with LLaMA.cpp and small language models.
