Erik Kristensen and Napalys Klicius write about four changes to the GitHub Copilot harness that reduce token costs without sacrificing task quality. The central insight is that optimizing individual tool calls is the wrong metric — a shorter response can cost more overall if it forces the agent to rerun commands or reread output. The four changes are: selectively compressing repetitive build/test/install output while preserving source-like content, removing unused line-number prefixes from file reads, halving the task-tool prompt via a meta-prompting loop, and batching background completion notifications so results arrive without an extra retrieval turn. Each was validated through offline agentic benchmarks and controlled online A/B experiments before shipping.
- RTK (Rust Token Killer) was evaluated and found to increase end-to-end cost despite shortening individual responses, because the agent reopened or reran commands to recover omitted details.
- The prompt compression initially caused a regression that offline tests missed: cautious parallelism guidance was rewritten into a hard scheduling policy, serializing independent agents. The fix was a single sentence: "Independent agents can run in parallel; consider side effects."
- A tighter file-tool instruction set that worked in Copilot code review actually increased cost in Copilot CLI, illustrating that evidence is local to the workload.
- The changes ship across all Copilot products sharing the same harness (CLI, app, code review); code review separately saw ~20% cost reduction from a prior migration to shared file tools.
- RTK (Rust Token Killer) was evaluated and found to increase end-to-end cost despite shortening individual responses, because the agent reopened or reran commands to recover omitted details.
- The prompt compression initially caused a regression that offline tests missed: cautious parallelism guidance was rewritten into a hard scheduling policy, serializing independent agents. The fix was a single sentence: "Independent agents can run in parallel; consider side effects."
- A tighter file-tool instruction set that worked in Copilot code review actually increased cost in Copilot CLI, illustrating that evidence is local to the workload.
- The changes ship across all Copilot products sharing the same harness (CLI, app, code review); code review separately saw ~20% cost reduction from a prior migration to shared file tools.
