Andrej Karpathy, Google, and Garry Tan are all adopting Markdown as the primary format for agent memory and organizational context. While Karpathy focuses on personal knowledge, Google targets enterprise runbooks, and Tan defines agent roles, they all share a reliance on versioned Markdown files. This shift suggests that the strategic advantage is moving from the specific large language model used to the curated knowledge base a team accumulates.
- Karpathy's LLM Wiki for personal knowledge bases
- Google's Open Knowledge Format for enterprise context
- Garry Tan's gstack for defining agent roles
- The migration of the competitive moat from models to portable data files
- Karpathy's LLM Wiki for personal knowledge bases
- Google's Open Knowledge Format for enterprise context
- Garry Tan's gstack for defining agent roles
- The migration of the competitive moat from models to portable data files
This article details Andrej Karpathy’s innovative approach to managing knowledge for AI projects, dubbed "LLM Knowledge Bases." This system aims to overcome the limitations of traditional Retrieval-Augmented Generation (RAG) and the frustrating context limits of "stateless" AI development.
**Key takeaways:**
* **Beyond RAG:** Karpathy proposes an alternative to vector databases and RAG, utilizing the LLM itself as a constantly updating "research librarian."
* **Markdown as Core:** The system centers around maintaining a structured knowledge base using Markdown files, which are easily readable, editable, and auditable.
* **Three-Stage Process:** The system involves: 1) **Data Ingest** (raw data to Markdown), 2) **Compilation** (LLM generates summaries, backlinks, and a structured wiki), and 3) **Active Maintenance** (LLM "lints" the wiki for consistency and new connections).
* **Self-Healing & Auditable:** The LLM actively maintains the knowledge base, ensuring it's self-healing and providing full traceability of information.
* **Enterprise Potential:** This approach could be a game-changer for businesses struggling with unstructured data, allowing them to create a dynamic, "Company Bible" of knowledge.
* **Scaling & Future:** While currently a "hacky collection of scripts," the system shows promise for scaling, potentially leading to synthetic data generation and fine-tuning of custom AI models.
The article highlights a shift towards treating LLMs not just as tools to *access* knowledge, but as agents actively *managing* and *improving* it. This philosophy prioritizes a "file-over-app" approach, giving users ownership of their data.
**Key takeaways:**
* **Beyond RAG:** Karpathy proposes an alternative to vector databases and RAG, utilizing the LLM itself as a constantly updating "research librarian."
* **Markdown as Core:** The system centers around maintaining a structured knowledge base using Markdown files, which are easily readable, editable, and auditable.
* **Three-Stage Process:** The system involves: 1) **Data Ingest** (raw data to Markdown), 2) **Compilation** (LLM generates summaries, backlinks, and a structured wiki), and 3) **Active Maintenance** (LLM "lints" the wiki for consistency and new connections).
* **Self-Healing & Auditable:** The LLM actively maintains the knowledge base, ensuring it's self-healing and providing full traceability of information.
* **Enterprise Potential:** This approach could be a game-changer for businesses struggling with unstructured data, allowing them to create a dynamic, "Company Bible" of knowledge.
* **Scaling & Future:** While currently a "hacky collection of scripts," the system shows promise for scaling, potentially leading to synthetic data generation and fine-tuning of custom AI models.
The article highlights a shift towards treating LLMs not just as tools to *access* knowledge, but as agents actively *managing* and *improving* it. This philosophy prioritizes a "file-over-app" approach, giving users ownership of their data.
