Zhang writes about Agora, a system that repurposes Git as shared memory for fleets of autonomous research agents, storing their contributions as an append-only directed acyclic graph where every claim is an immutable commit with parent edges encoding dependencies. In a 12-day run, 13 language-model workers with no assigned tasks or central planner tackled a weight-transfer problem—initializing a frozen 119.6M-parameter attention-SSM hybrid from 141 pretrained donor models without training data or gradient updates—and published 1,703 contributions, closing 62% of the gap to a trained GPT-2 124M (3.39 → 1.899 bits per byte). The winning recipe compresses donor next-token statistics into the target's embedding and output head, then adds short-range context through sparse edits to attention, feed-forward, and state-space blocks; its 145-commit ancestry spans 15 accounts and was independently reproduced 165 times with zero failures.
- A single mid-run human intervention was required to break a monoculture that the diversity-aware selection rule alone could not prevent
- A derived index exposes the frontier, neglected branches, and per-claim verification status
- The target's dimensions match no donor, making direct weight transfer impossible
- The authors acknowledge the experiment does not yet establish whether shared research state improves discovery per unit of compute and outline the controlled comparison that would settle this
- A single mid-run human intervention was required to break a monoculture that the diversity-aware selection rule alone could not prevent
- A derived index exposes the frontier, neglected branches, and per-claim verification status
- The target's dimensions match no donor, making direct weight transfer impossible
- The authors acknowledge the experiment does not yet establish whether shared research state improves discovery per unit of compute and outline the controlled comparison that would settle this
pi-autoresearch is an autonomous experiment loop for optimizing various targets like test speed, bundle size, LLM training, or build times. Inspired by karpathy/autoresearch, it utilizes a skill-extension architecture, allowing domain-agnostic infrastructure paired with domain-specific knowledge. The core workflow involves editing code, committing changes, running experiments, logging results, and either keeping or reverting the changes – a cycle that repeats indefinitely. Key components include a status widget, a detailed dashboard, and configuration options for customizing behavior. It persists experiment data in `autoresearch.jsonl` and session context in `autoresearch.md` for resilience and reproducibility.
This project, `autoresearch-opencode`, is an autonomous experiment loop designed for use with OpenCode. It's a port of `pi-autoresearch`, but implemented as a pure skill, eliminating the need for an MCP server and relying solely on instructions the agent follows using its built-in tools. The skill allows users to automate optimization tasks, as demonstrated by the example of optimizing the BogoSort algorithm which achieved a 7,802x speedup by leveraging Python's `bisect` module for sorted-state detection.
The system maintains state using a JSONL file, enabling resume/pause functionality and detailed experiment tracking. It provides a dashboard for monitoring progress and ensures data integrity through atomic writes and validation checks.
The system maintains state using a JSONL file, enabling resume/pause functionality and detailed experiment tracking. It provides a dashboard for monitoring progress and ensures data integrity through atomic writes and validation checks.
The article details “autoresearch,” a project by Karpathy where an AI agent autonomously experiments with training a small language model (nanochat) to improve its performance. The agent modifies the `train.py` file, trains for a fixed 5-minute period, and evaluates the results, repeating this process to iteratively refine the model. The project aims to demonstrate autonomous AI research, focusing on a simplified, single-GPU setup with a clear metric (validation bits per byte).
* **Autonomous Research:** The core concept of AI-driven experimentation.
* **nanochat:** The small language model used for training.
* **Fixed Time Budget:** Each experiment runs for exactly 5 minutes.
* **program.md:** The file containing instructions for the AI agent.
* **Single-File Modification:** The agent only edits `train.py`.
* **Autonomous Research:** The core concept of AI-driven experimentation.
* **nanochat:** The small language model used for training.
* **Fixed Time Budget:** Each experiment runs for exactly 5 minutes.
* **program.md:** The file containing instructions for the AI agent.
* **Single-File Modification:** The agent only edits `train.py`.
