Pedro Cuenca writes Meta released Muse Glimmer-30B, a local, open-source multimodal model distilled from its larger Muse architecture. Designed for agentic workflows, it combines a 28B text decoder with a 2B vision encoder, supporting image, video, and multimodal tool calling out of the box. The release includes immediate compatibility with major inference frameworks like transformers, llama.cpp, and vLLM, alongside built-in speculative decoding for faster generation.
- Features a hybrid attention pattern alternating between three sliding window layers and one full attention layer.
- Incorporates a DFlash block-diffusion drafter to accelerate structured text generation like coding.
- Supports fine-tuning via TRL with practical minimums ranging from one to eight H100 GPUs depending on the method.
- Demonstrates autonomous agent capabilities such as self-quantization, self-deployment, and hardware-specific optimization.
- Features a hybrid attention pattern alternating between three sliding window layers and one full attention layer.
- Incorporates a DFlash block-diffusion drafter to accelerate structured text generation like coding.
- Supports fine-tuning via TRL with practical minimums ranging from one to eight H100 GPUs depending on the method.
- Demonstrates autonomous agent capabilities such as self-quantization, self-deployment, and hardware-specific optimization.
The author explores the utility of Google DeepMind's Gemma 4 as a powerful option for running large language models locally on consumer hardware. By testing the E4B variant using tools like LM Studio and llama.cpp, they demonstrate how open-weight models can handle multimodal tasks including text, image analysis, and audio processing with impressive precision and privacy.
This document details how to run Google's Gemma 4 models locally, including the E2B, E4B, 26B-A4B, and 31B variants. Gemma 4 is a family of open models supporting over 140 languages and up to 256K context, available in both dense and MoE configurations. The E2B and E4B models support image and audio input. These models can be run locally on your device and fine-tuned using Unsloth Studio. The document outlines hardware requirements, recommended settings, and best practices for prompting and multimodal use, including guidance on context length and thinking mode.
