Alex Monahan writes that the open-weight Qwen 3.8 27B model, running locally via LM Studio on a consumer laptop, achieves frontier-level agentic SQL performance at essentially zero marginal cost. On the DABstep benchmark (400+ questions), the locally-run 4-bit quantized model outperformed OpenAI's GPT 5.6 Luna Max at 17x lower cost, and a 3-bit quant still worked on a nearly five-year-old M1 Pro MacBook Pro with only 16GB RAM. The setup pairs the local LLM with DuckDB for query execution, with MotherDuck's cloud hypertenancy as an optional escape hatch for scale.
- MTP (Multi-Token Prediction) yields ~30% throughput boost on M5 hardware but actually slows down older M1 Pro chips
- Runtime remains the main gap: 5–6 min per question locally vs. 25–40 sec for cloud frontier models
- The benchmark context layer was built using a frontier model (Claude Fable 5); only the eval loop runs locally
- Roughly 1 in 30 Macs in the wild have the 16GB+ unified memory needed; most laptops don't qualify
- Including laptop depreciation, the cost rises to ~$6 per 1,000 questions answered
- MTP (Multi-Token Prediction) yields ~30% throughput boost on M5 hardware but actually slows down older M1 Pro chips
- Runtime remains the main gap: 5–6 min per question locally vs. 25–40 sec for cloud frontier models
- The benchmark context layer was built using a frontier model (Claude Fable 5); only the eval loop runs locally
- Roughly 1 in 30 Macs in the wild have the 16GB+ unified memory needed; most laptops don't qualify
- Including laptop depreciation, the cost rises to ~$6 per 1,000 questions answered
Adam Conway writes that Qwen 3.8 27B, a 17 GB open-weights model running on a single Lenovo ThinkStation PGX, completed a commercial app's license reverse-engineering task in roughly 30 minutes'', statically disassembling arm64 code, recovering a deliberately obscured RSA public key, self-correcting an initial hash mismatch, and producing a working authentication bypass entirely offline.
- The model detected the jailbreak prompt, correctly identified the real developer from the signing certificate, then proceeded to audit the scheme and ultimately built the bypass on its own
- The target app's licensing was described as unusually thorough for its class: one-time online activation, offline signature check, hardware serial binding, embedded revocation list, and signed update path
- The model ran via the Pi harness using only standard Bash-based tools; no execution of the target app occurred until the final bypass demonstration
- Default max reasoning effort means even trivial requests consume hundreds to thousands of tokens, making the self-correction loop verbose but accurate
- Conway frames this as a shift in the threat model: a capable local model with no cloud oversight gives full control to whoever is at the keyboard
- The model detected the jailbreak prompt, correctly identified the real developer from the signing certificate, then proceeded to audit the scheme and ultimately built the bypass on its own
- The target app's licensing was described as unusually thorough for its class: one-time online activation, offline signature check, hardware serial binding, embedded revocation list, and signed update path
- The model ran via the Pi harness using only standard Bash-based tools; no execution of the target app occurred until the final bypass demonstration
- Default max reasoning effort means even trivial requests consume hundreds to thousands of tokens, making the self-correction loop verbose but accurate
- Conway frames this as a shift in the threat model: a capable local model with no cloud oversight gives full control to whoever is at the keyboard
