graphify is an AI coding assistant skill that transforms codebases, documents, and images into a structured, queryable knowledge graph. By utilizing deterministic AST parsing via tree-sitter for code and multimodal LLM capabilities for unstructured data like PDFs and screenshots, it creates a comprehensive map of concepts and relationships. This allows developers to understand complex architectures faster and find the "why" behind design decisions. A key advantage is its massive reduction in token usage per query compared to reading raw files, making it highly efficient for large-scale projects. The tool supports 19 programming languages and integrates seamlessly with platforms like Claude Code and Codex, providing an interactive, persistent, and highly organized way to navigate any codebase or research corpus.
This article details Andrej Karpathy’s innovative approach to managing knowledge for AI projects, dubbed "LLM Knowledge Bases." This system aims to overcome the limitations of traditional Retrieval-Augmented Generation (RAG) and the frustrating context limits of "stateless" AI development.
**Key takeaways:**
* **Beyond RAG:** Karpathy proposes an alternative to vector databases and RAG, utilizing the LLM itself as a constantly updating "research librarian."
* **Markdown as Core:** The system centers around maintaining a structured knowledge base using Markdown files, which are easily readable, editable, and auditable.
* **Three-Stage Process:** The system involves: 1) **Data Ingest** (raw data to Markdown), 2) **Compilation** (LLM generates summaries, backlinks, and a structured wiki), and 3) **Active Maintenance** (LLM "lints" the wiki for consistency and new connections).
* **Self-Healing & Auditable:** The LLM actively maintains the knowledge base, ensuring it's self-healing and providing full traceability of information.
* **Enterprise Potential:** This approach could be a game-changer for businesses struggling with unstructured data, allowing them to create a dynamic, "Company Bible" of knowledge.
* **Scaling & Future:** While currently a "hacky collection of scripts," the system shows promise for scaling, potentially leading to synthetic data generation and fine-tuning of custom AI models.
The article highlights a shift towards treating LLMs not just as tools to *access* knowledge, but as agents actively *managing* and *improving* it. This philosophy prioritizes a "file-over-app" approach, giving users ownership of their data.
**Key takeaways:**
* **Beyond RAG:** Karpathy proposes an alternative to vector databases and RAG, utilizing the LLM itself as a constantly updating "research librarian."
* **Markdown as Core:** The system centers around maintaining a structured knowledge base using Markdown files, which are easily readable, editable, and auditable.
* **Three-Stage Process:** The system involves: 1) **Data Ingest** (raw data to Markdown), 2) **Compilation** (LLM generates summaries, backlinks, and a structured wiki), and 3) **Active Maintenance** (LLM "lints" the wiki for consistency and new connections).
* **Self-Healing & Auditable:** The LLM actively maintains the knowledge base, ensuring it's self-healing and providing full traceability of information.
* **Enterprise Potential:** This approach could be a game-changer for businesses struggling with unstructured data, allowing them to create a dynamic, "Company Bible" of knowledge.
* **Scaling & Future:** While currently a "hacky collection of scripts," the system shows promise for scaling, potentially leading to synthetic data generation and fine-tuning of custom AI models.
The article highlights a shift towards treating LLMs not just as tools to *access* knowledge, but as agents actively *managing* and *improving* it. This philosophy prioritizes a "file-over-app" approach, giving users ownership of their data.
pi-autoresearch is an autonomous experiment loop for optimizing various targets like test speed, bundle size, LLM training, or build times. Inspired by karpathy/autoresearch, it utilizes a skill-extension architecture, allowing domain-agnostic infrastructure paired with domain-specific knowledge. The core workflow involves editing code, committing changes, running experiments, logging results, and either keeping or reverting the changes – a cycle that repeats indefinitely. Key components include a status widget, a detailed dashboard, and configuration options for customizing behavior. It persists experiment data in `autoresearch.jsonl` and session context in `autoresearch.md` for resilience and reproducibility.
This project, `autoresearch-opencode`, is an autonomous experiment loop designed for use with OpenCode. It's a port of `pi-autoresearch`, but implemented as a pure skill, eliminating the need for an MCP server and relying solely on instructions the agent follows using its built-in tools. The skill allows users to automate optimization tasks, as demonstrated by the example of optimizing the BogoSort algorithm which achieved a 7,802x speedup by leveraging Python's `bisect` module for sorted-state detection.
The system maintains state using a JSONL file, enabling resume/pause functionality and detailed experiment tracking. It provides a dashboard for monitoring progress and ensures data integrity through atomic writes and validation checks.
The system maintains state using a JSONL file, enabling resume/pause functionality and detailed experiment tracking. It provides a dashboard for monitoring progress and ensures data integrity through atomic writes and validation checks.
The article details “autoresearch,” a project by Karpathy where an AI agent autonomously experiments with training a small language model (nanochat) to improve its performance. The agent modifies the `train.py` file, trains for a fixed 5-minute period, and evaluates the results, repeating this process to iteratively refine the model. The project aims to demonstrate autonomous AI research, focusing on a simplified, single-GPU setup with a clear metric (validation bits per byte).
* **Autonomous Research:** The core concept of AI-driven experimentation.
* **nanochat:** The small language model used for training.
* **Fixed Time Budget:** Each experiment runs for exactly 5 minutes.
* **program.md:** The file containing instructions for the AI agent.
* **Single-File Modification:** The agent only edits `train.py`.
* **Autonomous Research:** The core concept of AI-driven experimentation.
* **nanochat:** The small language model used for training.
* **Fixed Time Budget:** Each experiment runs for exactly 5 minutes.
* **program.md:** The file containing instructions for the AI agent.
* **Single-File Modification:** The agent only edits `train.py`.
As generative AI reshapes software development, natural language commands are replacing traditional programming syntax, but experts question if English can ever match the precision of code.
AI pioneer Andrej Karpathy proposes an AI application that could enhance the e-book reading experience with interactive features. Amazon is reportedly looking for an applied scientist to improve Kindle's reading and publishing experience using AI.
Andrej Karpathy's recommended paper reading list, covering various aspects of Language Models (LLMs), including attention mechanisms, unsupervised multi-task learning (GPT-2), instruction-following language models (InstructGPT), LLaMA, reinforcement learning from human feedback (RLAIF), and early experiments of GPT-4, offering insights into significant research developments in LLM and their role in AI landscape, benefiting both novice and experienced AI enthusiasts.
"Refreshing my understanding of deep learning as a "stack of data transformations" is incredibly powerful. It's like a sequence of layers, each layer transforming the input data into something more abstract and informative. This perspective makes it easier to understand how neural networks process information."
