This article examines the architectural implications of choosing between stateless and stateful designs when building agentic systems. It evaluates how an agent's approach to managing memory impacts deployment, horizontal scaling, and client-side complexity.
- Stateless agents allow for easy horizontal scaling since no user memory is stored on a backend server, but they require the client to send the full conversation history with every request, leading to increased token usage as conversations grow.
- Stateful agents manage their own context through a database layer using session identifiers, which simplifies client interactions and supports complex workflows, though it introduces challenges in distributed scaling and data persistence.
- Stateless agents allow for easy horizontal scaling since no user memory is stored on a backend server, but they require the client to send the full conversation history with every request, leading to increased token usage as conversations grow.
- Stateful agents manage their own context through a database layer using session identifiers, which simplifies client interactions and supports complex workflows, though it introduces challenges in distributed scaling and data persistence.
Loop Engineering for RAG Generation: An LLM Cascade from a Cheap Local Model Up to a Hosted FlagshipTwo angles on the cascade, cost and a validation loop, backed by a real sweep of twenty local models against a hosted flagship
Simon Willison reviews Ornith-1.0, a new series of open-weights models from DeepReinforce designed for agentic coding. Built on Gemma 4 and Qwen 3.5, these models range from 9B to 397B parameters and demonstrate high proficiency in coding benchmarks and tool-use tasks.
- Available in 9B Dense, 31B Dense, 35B MoE, and 397B MoE variants
- Capable of navigating codebases and executing complex tool calls
- Built on Apache 2.0 licensed base models
- Available in 9B Dense, 31B Dense, 35B MoE, and 397B MoE variants
- Capable of navigating codebases and executing complex tool calls
- Built on Apache 2.0 licensed base models
The article argues that giving AI agents access to flexible command-line interfaces is more efficient than using many specialized MCP servers because it reduces context window bloat and enables powerful tool composition through piping.
* Specialized tools often clutter the model's context, while a single CLI allows data to flow directly between processes without increasing token costs.
* Terminal access facilitates complex workflows like multi-environment tasks and cross-tool automation that are cumbersome with individual MCP servers.
* Although providing terminal access increases security risks, these can be managed through sandboxing and restricted user permissions.
* Specialized tools often clutter the model's context, while a single CLI allows data to flow directly between processes without increasing token costs.
* Terminal access facilitates complex workflows like multi-environment tasks and cross-tool automation that are cumbersome with individual MCP servers.
* Although providing terminal access increases security risks, these can be managed through sandboxing and restricted user permissions.
The author proposes a 5-layer framework to standardize "harness engineering":
1. **Constraint (Architecture):** Deterministic rules (linters, API contracts).
2. **Context (Dev):** Memory and knowledge injection.
3. **Execution (Platform):** Tool orchestration and sandboxing.
4. **Verification (Dev/QA):** Output validation and error loops.
5. **Lifecycle (SRE):** Monitoring, cost tracking, and recovery.
**Strategic Insight:** While platforms like Anthropic are increasingly absorbing the Context, Execution, and Lifecycle layers, developers must still own **Constraint** and **Verification**. To maximize efficiency on managed platforms, teams should prioritize deterministic constraints (Layer 1) to reduce token waste and improve reliability.
1. **Constraint (Architecture):** Deterministic rules (linters, API contracts).
2. **Context (Dev):** Memory and knowledge injection.
3. **Execution (Platform):** Tool orchestration and sandboxing.
4. **Verification (Dev/QA):** Output validation and error loops.
5. **Lifecycle (SRE):** Monitoring, cost tracking, and recovery.
**Strategic Insight:** While platforms like Anthropic are increasingly absorbing the Context, Execution, and Lifecycle layers, developers must still own **Constraint** and **Verification**. To maximize efficiency on managed platforms, teams should prioritize deterministic constraints (Layer 1) to reduce token waste and improve reliability.
This paper introduces Natural-Language Agent Harnesses (NLAHs) – a new approach to AI agent harness design. NLAHs use editable natural language, improving portability and study, unlike traditional code-embedded harnesses. The authors also present the Intelligent Harness Runtime (IHR) and demonstrate viability through coding/computer-use benchmarks.
