Maria Deutscher writes that Z.ai has open-sourced GLM-5.3-Flash, a 320-billion-parameter mixture-of-experts LLM that debuted the prior week under the anonymous codename "Ox Alpha" on OpenRouter, sparking industry speculation about its origin. The model is 10x more cost-efficient than Z.ai's predecessor, using sparse and linear attention mechanisms to dramatically reduce memory and processing overhead while supporting a 1-million-token context window.
- Linear attention replaces the softmax function, cutting memory scaling from quadratic to linear as prompt size grows
- Trained on 30 trillion tokens using an mHC technique that prevents gradient distortion through neuron layers
- Scored highest among Claude Opus 4.8, GPT-5.6 Terra, and Gemini 3.7 Flash on GDPval-AA v2, a knowledge-work evaluation
- Linear attention replaces the softmax function, cutting memory scaling from quadratic to linear as prompt size grows
- Trained on 30 trillion tokens using an mHC technique that prevents gradient distortion through neuron layers
- Scored highest among Claude Opus 4.8, GPT-5.6 Terra, and Gemini 3.7 Flash on GDPval-AA v2, a knowledge-work evaluation
