Anubhab Banerjee writes that three Qwen2.5-Coder agents of different sizes can share pre-computed token IDs instead of re-tokenizing the same text, using a Google Open Knowledge Format-inspired Markdown+YAML hand-off with an added token_pointer field to /dev/shm, cutting median TTFT by 28% on 3B and 37.8% on 1.5B while avoiding KV-cache sharing.
- Uses /dev/shm/qwen_tokens/ with NumPy .npy int64 arrays for zero-copy hand-off
- Agents run as separate OS processes to release VRAM automatically between models
- Runtime guardrail verifies full ~151,936-entry get_vocab() dict equality across models
- OKF files add non-standard token_pointer and tokenizer_model_id fields for provenance
- Savings apply to input side only; each agent still tokenizes its own output
- Uses /dev/shm/qwen_tokens/ with NumPy .npy int64 arrays for zero-copy hand-off
- Agents run as separate OS processes to release VRAM automatically between models
- Runtime guardrail verifies full ~151,936-entry get_vocab() dict equality across models
- OKF files add non-standard token_pointer and tokenizer_model_id fields for provenance
- Savings apply to input side only; each agent still tokenizes its own output
