Asif Razzaq writes that Alibaba's Qwen team has released Qwen3.8-Flash-Next, an open-weight multimodal Mixture-of-Experts model pairing a 125B backbone with a 51B N-gram embedding table and a 4B multi-token prediction module, totaling 180B on disk but activating only 6B parameters per token. The architecture blends Gated DeltaNet linear-attention layers (three of every four) with Qwen Sparse Attention at micro-block granularity, and is positioned as the architectural preview of Qwen4 in the same role Qwen3-Next played for Qwen3.5. Training cost is reported at roughly one-ninth that of Qwen3.7-Plus.
- Licensed under qwen-community-1.0, not Apache-2.0; verify terms before commercial use.
- FP8 checkpoint is 172.78 GiB; minimum validated config is TP2 on GB300, so self-hosting requires a multi-GPU node.
- The 20M-entry bigram/trigram table at layer 2 can be offloaded to host memory with asynchronous prefetch (NVIDIA only).
- Claude Opus 4.6 (Max) still leads on HLE (40.0 vs. Qwen's 35.9); DeepSeek-V4-Flash-0731 leads NL2Repo-Bench (54.2 vs. 48.1).
- Native context is 262,144 tokens, extensible to 1,000,000 via YaRN.
- Licensed under qwen-community-1.0, not Apache-2.0; verify terms before commercial use.
- FP8 checkpoint is 172.78 GiB; minimum validated config is TP2 on GB300, so self-hosting requires a multi-GPU node.
- The 20M-entry bigram/trigram table at layer 2 can be offloaded to host memory with asynchronous prefetch (NVIDIA only).
- Claude Opus 4.6 (Max) still leads on HLE (40.0 vs. Qwen's 35.9); DeepSeek-V4-Flash-0731 leads NL2Repo-Bench (54.2 vs. 48.1).
- Native context is 262,144 tokens, extensible to 1,000,000 via YaRN.
