paoloap wrote companion code for a Medium article on agent harnesses, including five layers: execution boundary, sandboxing, memory persistence, verification loops, and context pipelines, each with failure and guard scripts running alongside.
Move enforcement out of system prompts to deterministic code; a rule like "never delete without human approval" becomes a pre-execution hook denying the call, not a sentence the model might forget.Every demo runs with no API key; the model is replaced by a ~40-line scripted stand-in that emits a fixed sequence of tool calls, making each failure reproducible and the CI suite able to assert behaviour rather than smoke-test it. A single dependency-free `harness.py` can be dropped into any existing agent loop to add boundary checks, path allowlists, hostname allowlists, a persistent store, read-only review, dry-run, and token-cost distillation.
- `boundary()` detects coroutine functions and awaits them, fixing a silent no-op that made the guard a no-op on the most common (async) agent loops
- `host_allowlist` compares the parsed hostname, so `api.openai.com.evil.com` and `evil.com/?x=api.openai.com` are both refused
- `Denied` subclasses `str` so it drops into the same slot a tool result occupies, keeping existing loops unchanged
- `boundary(rules, max_repeats=3)` detects identical consecutive denials and changes the message to break a deterministic deadlock at full token cost
- The repo is MIT-licensed, Python 3.8+, and has no external dependencies
Move enforcement out of system prompts to deterministic code; a rule like "never delete without human approval" becomes a pre-execution hook denying the call, not a sentence the model might forget.Every demo runs with no API key; the model is replaced by a ~40-line scripted stand-in that emits a fixed sequence of tool calls, making each failure reproducible and the CI suite able to assert behaviour rather than smoke-test it. A single dependency-free `harness.py` can be dropped into any existing agent loop to add boundary checks, path allowlists, hostname allowlists, a persistent store, read-only review, dry-run, and token-cost distillation.
- `boundary()` detects coroutine functions and awaits them, fixing a silent no-op that made the guard a no-op on the most common (async) agent loops
- `host_allowlist` compares the parsed hostname, so `api.openai.com.evil.com` and `evil.com/?x=api.openai.com` are both refused
- `Denied` subclasses `str` so it drops into the same slot a tool result occupies, keeping existing loops unchanged
- `boundary(rules, max_repeats=3)` detects identical consecutive denials and changes the message to break a deterministic deadlock at full token cost
- The repo is MIT-licensed, Python 3.8+, and has no external dependencies
A curated collection of resources, patterns, and templates for building reliable scaffolding for agents. Harness engineering is the discipline of designing the systems surrounding an agent—such as context delivery, tool interfaces, planning artifacts, verification loops, memory systems, and sandboxes—that determine its success or failure on real tasks. This focus is on the harness rather than the model.
- Design primitives for loops, planning, and memory
- Reference implementations and tutorials
- Security, sandboxing, and permissions
- Evaluation, verification, and observability
- Task runners and orchestration
- Human-in-the-loop and production operations
- Design primitives for loops, planning, and memory
- Reference implementations and tutorials
- Security, sandboxing, and permissions
- Evaluation, verification, and observability
- Task runners and orchestration
- Human-in-the-loop and production operations
