Simon Willison reviews Ornith-1.0, a new series of open-weights models from DeepReinforce designed for agentic coding. Built on Gemma 4 and Qwen 3.5, these models range from 9B to 397B parameters and demonstrate high proficiency in coding benchmarks and tool-use tasks.
- Available in 9B Dense, 31B Dense, 35B MoE, and 397B MoE variants
- Capable of navigating codebases and executing complex tool calls
- Built on Apache 2.0 licensed base models
- Available in 9B Dense, 31B Dense, 35B MoE, and 397B MoE variants
- Capable of navigating codebases and executing complex tool calls
- Built on Apache 2.0 licensed base models
An exploration of high-performing small language models with under 7 billion parameters that can run locally on consumer hardware like laptops and smartphones. The article explains how advancements in training data quality, model distillation from larger frontier models, and architectural improvements like Mixture-of-Experts have enabled these compact models to compete with much larger versions on reasoning benchmarks. It provides a curated guide of top available models on Hugging Face, detailing their specific strengths, benchmark performance, and providing Python code for implementation.
Key models covered:
- Qwen3.5-4B for multilingual tasks and long context windows
- Microsoft Phi-4-mini-instruct for reasoning-heavy English workloads
- Google Gemma 3 4B IT for coding and mathematics
- Google Gemma 3n E4B for efficient mobile and on-device deployment
- Meta Llama 3.2 3B Instruct for tool calling and community support
- SmolLM3-3B for research transparency and open-source projects
- DeepSeek-R1-Distill-Qwen-1.5B for lightweight reasoning on edge devices
- Qwen3-0.6B for ultra-constrained hardware and text classification
Key models covered:
- Qwen3.5-4B for multilingual tasks and long context windows
- Microsoft Phi-4-mini-instruct for reasoning-heavy English workloads
- Google Gemma 3 4B IT for coding and mathematics
- Google Gemma 3n E4B for efficient mobile and on-device deployment
- Meta Llama 3.2 3B Instruct for tool calling and community support
- SmolLM3-3B for research transparency and open-source projects
- DeepSeek-R1-Distill-Qwen-1.5B for lightweight reasoning on edge devices
- Qwen3-0.6B for ultra-constrained hardware and text classification
Needle is an experimental 26m parameter Simple Attention Network distilled from Gemini 3.1, designed to redefine tiny AI for consumer devices like phones, watches, and glasses. It specializes in single-shot function calling, outperforming larger models such as FunctionGemma-270m and Qwen-0.6B in that specific domain. The project provides fully open weights, dataset generation methods, and tools for local finetuning on consumer hardware.
* 26m parameter architecture optimized for extremely small devices.
* High performance prefill and decode speeds when running on Cactus.
* Pretrained on 200B tokens and post-trained for function calling.
* Support for local finetuning via a web UI playground or CLI.
* 26m parameter architecture optimized for extremely small devices.
* High performance prefill and decode speeds when running on Cactus.
* Pretrained on 200B tokens and post-trained for function calling.
* Support for local finetuning via a web UI playground or CLI.
Google DeepMind has released four new open-source, vision-capable LLMs under the Apache 2.0 license – Gemma 4, with sizes ranging from 2B to 31B parameters, and a 26B-A4B Mixture-of-Experts model. The models are notable for their intelligence-per-parameter ratio, with the smaller models (E2B and E4B) utilizing Per-Layer Embeddings to maximize efficiency.
The models support both vision and audio input, although audio functionality is not yet fully implemented in tools like LM Studio or Ollama. Testing with LM Studio showed varying results, with the 31B model experiencing output issues. The author also experimented with the models through the llm-gemini API, generating SVG images of a pelican riding a bicycle to assess their visual capabilities.
The models support both vision and audio input, although audio functionality is not yet fully implemented in tools like LM Studio or Ollama. Testing with LM Studio showed varying results, with the 31B model experiencing output issues. The author also experimented with the models through the llm-gemini API, generating SVG images of a pelican riding a bicycle to assess their visual capabilities.
This Hugging Face page details the Gemma 4 31B-it model, an open-weights multimodal model created by Google DeepMind. Gemma 4 can process both text and image inputs, generating text outputs, with smaller models also supporting audio. It comes in various sizes (E2B, E4B, 26B A4B, and 31B) allowing for deployment on diverse hardware, from phones to servers.
The model boasts a context window of up to 256K tokens and supports over 140 languages. It utilizes dense and Mixture-of-Experts (MoE) architectures, excelling in tasks like text generation, coding, and reasoning. The page provides details on model data, training, ethics, usage, limitations, and best practices, along with code snippets for getting started with Transformers.
The model boasts a context window of up to 256K tokens and supports over 140 languages. It utilizes dense and Mixture-of-Experts (MoE) architectures, excelling in tasks like text generation, coding, and reasoning. The page provides details on model data, training, ethics, usage, limitations, and best practices, along with code snippets for getting started with Transformers.
This article explains how to implement function calling with Google’s Gemma 3 27B model. It covers the concept of function calling, the step‑by‑step workflow, and provides a practical example using a Python `convert` function to turn $200,000 into EUR. The post walks through prompting Gemma, parsing its `tool_code` output, executing the function with `eval`, and returning a friendly final response. It also demonstrates how to set up the Google‑GenAI SDK, create a chat session, and extract tool calls. The discussion highlights Gemma’s multilingual, multimodal, and agentic capabilities, making it suitable for real‑world AI assistants that need to interact with external APIs and tools.
This article details research into finding the optimal architecture for small language models (70M parameters), exploring depth-width tradeoffs, comparing different architectures, and introducing Dhara-70M, a diffusion model offering 3.8x faster throughput with improved factuality.
This discussion details performance benchmarks of llama.cpp on an NVIDIA DGX Spark, including tests for various models (gpt-oss-20b, gpt-oss-120b, Qwen3, Qwen2.5, Gemma, GLM) with different context depths and batch sizes.
Extracting structured information effectively and accurately from long unstructured text with LangExtract and LLMs. This article explores Google’s LangExtract framework and its open-source LLM, Gemma 3, demonstrating how to parse an insurance policy to surface details like exclusions.
A detailed comparison of the architectures of recent large language models (LLMs) including DeepSeek-V3, OLMo 2, Gemma 3, Mistral Small 3.1, Llama 4, Qwen3, SmolLM3, and Kimi 2, focusing on key design choices and their impact on performance and efficiency.
1. **DeepSeek V3/R1**:
- Uses Multi-Head Latent Attention (MLA) and Mixture-of-Experts (MoE) for efficiency.
- MLA compresses key and value tensors to reduce KV cache memory usage.
- MoE activates only a subset of experts per token, improving inference efficiency.
2. **OLMo 2**:
- Focuses on transparency in training data and code.
- Uses RMSNorm layers placed after attention and feed-forward modules (Post-Norm).
- Introduces QK-Norm, an additional RMSNorm layer applied to queries and keys inside the attention mechanism.
3. **Gemma 3**:
- Employs sliding window attention to reduce memory requirements in the KV cache.
- Uses a 5:1 ratio of sliding window attention to global attention layers.
- Combines Pre-Norm and Post-Norm RMSNorm layers around the attention module.
4. **Mistral Small 3.1**:
- Outperforms Gemma 3 27B on several benchmarks while being faster.
- Uses a standard architecture with a custom tokenizer and reduced KV cache and layer count.
5. **Llama 4**:
- Adopts an MoE approach similar to DeepSeek V3 but with fewer, larger experts.
- Alternates MoE and dense modules in every other transformer block.
6. **Qwen3**:
- Comes in both dense and MoE variants.
- Dense models are easier to fine-tune and deploy, while MoE models are optimized for scaling inference.
7. **SmolLM3**:
- Uses No Positional Embeddings (NoPE), omitting explicit positional information injection.
- NoPE improves length generalization, meaning performance deteriorates less with increased sequence length.
8. **Kimi K2 and Kimi K2 Thinking**:
- Uses a variant of the Muon optimizer over AdamW.
- Kimi K2 Thinking extends the context size to 256k tokens.
9. **GPT-OSS**:
- OpenAI's first open-weight models since GPT-2.
- Uses sliding window attention and a width-versus-depth trade-off.
10. **Grok 2.5**:
- Uses a small number of large experts and a shared expert module.
- Reflects an older trend in MoE architectures.
11. **GLM-4.5**:
- Comes in two variants: a 355-billion-parameter model and a more compact 106-billion-parameter version.
- Uses a shared expert and starts with several dense layers before introducing MoE blocks.
12. **Qwen3-Next**:
- Introduces a Gated DeltaNet + Gated Attention hybrid mechanism.
- Uses Multi-Token Prediction (MTP) for efficiency.
13. **MiniMax-M2**:
- Uses per-layer QK-Norm and partial RoPE.
- More "sparse" than Qwen3, with fewer active experts per token.
14. **Kimi Linear**:
- Modifies the linear attention mechanism with Kimi Delta Attention (KDA).
- Combines Gated DeltaNet with Multi-Head Latent Attention (MLA).
15. **Olmo 3 Thinking**:
- Uses sliding window attention and YaRN for context extension.
- Comes in base, instruct, and reasoning variants.
16. **DeepSeek V3.2**:
- Adds a sparse attention mechanism to improve efficiency.
- On par with GPT-5.1 and Gemini 3.0 Pro on certain benchmarks.
17. **Mistral 3**:
- First MoE model since Mixtral in 2023.
- Partnered with NVIDIA for optimization on Blackwell chips.
18. **Nemotron 3**:
- A Transformer-Mamba hybrid architecture.
- Interleaves Mamba-2 sequence-modeling blocks with sparse MoE feed-forward layers.
19. **Xiaomi MiMo-V2-Flash**:
- Uses sliding window attention in a 5:1 ratio with global attention.
- Employs multi-token prediction (MTP) for efficiency.
20. **Arcee AI Trinity Large**:
- Uses alternating local:global attention layers, NoPE, and gated attention.
- Introduces depth-scaled sandwich norm for training stability.
1. **DeepSeek V3/R1**:
- Uses Multi-Head Latent Attention (MLA) and Mixture-of-Experts (MoE) for efficiency.
- MLA compresses key and value tensors to reduce KV cache memory usage.
- MoE activates only a subset of experts per token, improving inference efficiency.
2. **OLMo 2**:
- Focuses on transparency in training data and code.
- Uses RMSNorm layers placed after attention and feed-forward modules (Post-Norm).
- Introduces QK-Norm, an additional RMSNorm layer applied to queries and keys inside the attention mechanism.
3. **Gemma 3**:
- Employs sliding window attention to reduce memory requirements in the KV cache.
- Uses a 5:1 ratio of sliding window attention to global attention layers.
- Combines Pre-Norm and Post-Norm RMSNorm layers around the attention module.
4. **Mistral Small 3.1**:
- Outperforms Gemma 3 27B on several benchmarks while being faster.
- Uses a standard architecture with a custom tokenizer and reduced KV cache and layer count.
5. **Llama 4**:
- Adopts an MoE approach similar to DeepSeek V3 but with fewer, larger experts.
- Alternates MoE and dense modules in every other transformer block.
6. **Qwen3**:
- Comes in both dense and MoE variants.
- Dense models are easier to fine-tune and deploy, while MoE models are optimized for scaling inference.
7. **SmolLM3**:
- Uses No Positional Embeddings (NoPE), omitting explicit positional information injection.
- NoPE improves length generalization, meaning performance deteriorates less with increased sequence length.
8. **Kimi K2 and Kimi K2 Thinking**:
- Uses a variant of the Muon optimizer over AdamW.
- Kimi K2 Thinking extends the context size to 256k tokens.
9. **GPT-OSS**:
- OpenAI's first open-weight models since GPT-2.
- Uses sliding window attention and a width-versus-depth trade-off.
10. **Grok 2.5**:
- Uses a small number of large experts and a shared expert module.
- Reflects an older trend in MoE architectures.
11. **GLM-4.5**:
- Comes in two variants: a 355-billion-parameter model and a more compact 106-billion-parameter version.
- Uses a shared expert and starts with several dense layers before introducing MoE blocks.
12. **Qwen3-Next**:
- Introduces a Gated DeltaNet + Gated Attention hybrid mechanism.
- Uses Multi-Token Prediction (MTP) for efficiency.
13. **MiniMax-M2**:
- Uses per-layer QK-Norm and partial RoPE.
- More "sparse" than Qwen3, with fewer active experts per token.
14. **Kimi Linear**:
- Modifies the linear attention mechanism with Kimi Delta Attention (KDA).
- Combines Gated DeltaNet with Multi-Head Latent Attention (MLA).
15. **Olmo 3 Thinking**:
- Uses sliding window attention and YaRN for context extension.
- Comes in base, instruct, and reasoning variants.
16. **DeepSeek V3.2**:
- Adds a sparse attention mechanism to improve efficiency.
- On par with GPT-5.1 and Gemini 3.0 Pro on certain benchmarks.
17. **Mistral 3**:
- First MoE model since Mixtral in 2023.
- Partnered with NVIDIA for optimization on Blackwell chips.
18. **Nemotron 3**:
- A Transformer-Mamba hybrid architecture.
- Interleaves Mamba-2 sequence-modeling blocks with sparse MoE feed-forward layers.
19. **Xiaomi MiMo-V2-Flash**:
- Uses sliding window attention in a 5:1 ratio with global attention.
- Employs multi-token prediction (MTP) for efficiency.
20. **Arcee AI Trinity Large**:
- Uses alternating local:global attention layers, NoPE, and gated attention.
- Introduces depth-scaled sandwich norm for training stability.
