Qwen publishes official GGUF weights for its 8-billion-parameter vision-language model, Qwen3-VL-8B-Instruct, allowing local inference of image, video, and text workloads through common tools like llama.cpp, Ollama, LM Studio, and vLLM without needing a custom Transformers environment. The model supports a 256K-token context window, 32-language OCR, 2D/3D grounding, GUI agent capabilities, and video timestamp localization, all under an Apache 2.0 license.
- The language model is separated from a multimodal projector (mmproj) file, letting users mix precisions such as a 4-bit language model with an F16 projector for better OCR accuracy
- The Q4_K_M language model plus a quantized projector can run on 16 GB unified-memory systems, though the full 256K context window is impractical on that hardware
- Benchmark scores include DocVQA 96.1, MMMU 69.6, and MMLU-Pro 68.6
- The language model is separated from a multimodal projector (mmproj) file, letting users mix precisions such as a 4-bit language model with an F16 projector for better OCR accuracy
- The Q4_K_M language model plus a quantized projector can run on 16 GB unified-memory systems, though the full 256K context window is impractical on that hardware
- Benchmark scores include DocVQA 96.1, MMMU 69.6, and MMLU-Pro 68.6
