A comprehensive technical guide on setting up a high-performance local large language model environment for agentic coding tasks. The author demonstrates how to run a quantized Qwen3.5-27B model on a remote RTX 4090 workstation and access it from a MacBook using Tailscale, integrating the setup with OpenCode and Codex.
Key topics include:
* Step-by-step llama.cpp build configuration for CUDA support.
* Using Tailscale to create a secure network between client and GPU machine.
* Optimizing VRAM usage through specific quantization (UD-Q4_K_XL) and context size management.
* Implementing a corrected chat template to prevent tool-calling errors in agentic workflows.
* Performance insights regarding hybrid architectures and KV cache precision.
Key topics include:
* Step-by-step llama.cpp build configuration for CUDA support.
* Using Tailscale to create a secure network between client and GPU machine.
* Optimizing VRAM usage through specific quantization (UD-Q4_K_XL) and context size management.
* Implementing a corrected chat template to prevent tool-calling errors in agentic workflows.
* Performance insights regarding hybrid architectures and KV cache precision.
This article discusses the author's experience setting up reverse proxies for self-hosted services, finding the process surprisingly straightforward despite extensive and often overwhelming documentation. It compares several popular options like Nginx, Traefik, Caddy, Envoy, SWAG, and HAProxy, ultimately recommending Caddy for its simplicity and features. It also touches on the relative ease of reverse proxy setup compared to configuring the services they front.
This article details how the author successfully ran OpenAI's Codex CLI against a gpt-oss:120b model hosted on an NVIDIA DGX Spark, accessed through a Tailscale network. It covers the setup of Tailscale, Ollama configuration, and the process of running the Codex CLI with the remote model, including building a Space Invaders game.
