Paolo Perrone states the code "harness" around an LLM agent affects cost more than capability, breaking it into five layers: execution boundary, sandboxing, memory persistence, verification loops, and context pipelines.
The agent loop is six lines of Python; engineering determines what the model does, accesses, retains, checks, and sees, affecting token spend and unsanctioned actions.
A June 2026 preprint measured up to a 40x difference in tokens per solved task across three harnesses on the same model, while pass-rate gaps were 0-8 percentage points with confidence intervals crossing zero.
- The motivating anecdote: a prompt rule forbidding unapproved edits was violated 76 times in one afternoon; rewriting it as a pre-dispatch hook that returns a deny eliminated the failures entirely.
A denylist sandbox is defeated by path traversal (`work/../secrets/api_key`), whereas an allowlist combined with `os.path.normpath` before comparison catches every spelling of the same file.
The author notes six issues: no peer-reviewed harness study by August 2026, harness gains don't generalize to new tasks, a fine-tuning success contradicted its reward signal, harnesses aren't portable across models, the field lacks a definition, and harness effects decrease with better base models.
- All five layers ship as runnable Python scripts in a public repo (github.com/paoloap-py/agent-harness-guide) that use a scripted stand-in for the model, so every failure mode is reproducible without an API key.
The agent loop is six lines of Python; engineering determines what the model does, accesses, retains, checks, and sees, affecting token spend and unsanctioned actions.
A June 2026 preprint measured up to a 40x difference in tokens per solved task across three harnesses on the same model, while pass-rate gaps were 0-8 percentage points with confidence intervals crossing zero.
- The motivating anecdote: a prompt rule forbidding unapproved edits was violated 76 times in one afternoon; rewriting it as a pre-dispatch hook that returns a deny eliminated the failures entirely.
A denylist sandbox is defeated by path traversal (`work/../secrets/api_key`), whereas an allowlist combined with `os.path.normpath` before comparison catches every spelling of the same file.
The author notes six issues: no peer-reviewed harness study by August 2026, harness gains don't generalize to new tasks, a fine-tuning success contradicted its reward signal, harnesses aren't portable across models, the field lacks a definition, and harness effects decrease with better base models.
- All five layers ship as runnable Python scripts in a public repo (github.com/paoloap-py/agent-harness-guide) that use a scripted stand-in for the model, so every failure mode is reproducible without an API key.
