An exploration of high-performing small language models with under 7 billion parameters that can run locally on consumer hardware like laptops and smartphones. The article explains how advancements in training data quality, model distillation from larger frontier models, and architectural improvements like Mixture-of-Experts have enabled these compact models to compete with much larger versions on reasoning benchmarks. It provides a curated guide of top available models on Hugging Face, detailing their specific strengths, benchmark performance, and providing Python code for implementation.
Key models covered:
- Qwen3.5-4B for multilingual tasks and long context windows
- Microsoft Phi-4-mini-instruct for reasoning-heavy English workloads
- Google Gemma 3 4B IT for coding and mathematics
- Google Gemma 3n E4B for efficient mobile and on-device deployment
- Meta Llama 3.2 3B Instruct for tool calling and community support
- SmolLM3-3B for research transparency and open-source projects
- DeepSeek-R1-Distill-Qwen-1.5B for lightweight reasoning on edge devices
- Qwen3-0.6B for ultra-constrained hardware and text classification
Key models covered:
- Qwen3.5-4B for multilingual tasks and long context windows
- Microsoft Phi-4-mini-instruct for reasoning-heavy English workloads
- Google Gemma 3 4B IT for coding and mathematics
- Google Gemma 3n E4B for efficient mobile and on-device deployment
- Meta Llama 3.2 3B Instruct for tool calling and community support
- SmolLM3-3B for research transparency and open-source projects
- DeepSeek-R1-Distill-Qwen-1.5B for lightweight reasoning on edge devices
- Qwen3-0.6B for ultra-constrained hardware and text classification
