Google has released Multi-Token Prediction (MTP) drafters for the Gemma 4 model family to significantly accelerate inference speeds. By utilizing a specialized speculative decoding architecture, these drafters can deliver up to a 3x speedup without compromising output quality or reasoning capabilities. This technology addresses memory-bandwidth bottlenecks by allowing a lightweight drafter to predict multiple future tokens that are then verified in parallel by the larger target model.
Key points:
* Improved responsiveness for real-time chat, voice applications, and agentic workflows.
* Faster local development on personal computers and consumer GPUs.
* Enhanced performance and battery efficiency on edge devices.
* Architectural optimizations including KV cache sharing and activation utilization.
* Available now under the Apache 2.0 license via Hugging Face and Kaggle.
Key points:
* Improved responsiveness for real-time chat, voice applications, and agentic workflows.
* Faster local development on personal computers and consumer GPUs.
* Enhanced performance and battery efficiency on edge devices.
* Architectural optimizations including KV cache sharing and activation utilization.
* Available now under the Apache 2.0 license via Hugging Face and Kaggle.
AMD now supports Google’s Gemma 4 models (2B–31B parameters) across its entire hardware lineup, including Instinct GPUs (datacenters), Radeon GPUs (workstations), and Ryzen AI processors (PCs). The integration is compatible with vLLM, SGLang, llama.cpp, Ollama, and Lemonade Server, aiming to optimize AI performance for both cloud and local deployment.
This GitHub repository details the "Restaurant Roulette" skill, a tool designed to help users discover restaurants based on their preferred cuisine and location. The skill functions by searching for up to 10 restaurants that match the specified criteria and presenting them in a spin wheel format, adding an element of fun to the dining decision-making process. The project is licensed under the Apache License, Version 2.0, promoting open-source collaboration and usage. It's part of the Google AI Edge Gallery, showcasing practical AI applications.
Google DeepMind has released four new open-source, vision-capable LLMs under the Apache 2.0 license – Gemma 4, with sizes ranging from 2B to 31B parameters, and a 26B-A4B Mixture-of-Experts model. The models are notable for their intelligence-per-parameter ratio, with the smaller models (E2B and E4B) utilizing Per-Layer Embeddings to maximize efficiency.
The models support both vision and audio input, although audio functionality is not yet fully implemented in tools like LM Studio or Ollama. Testing with LM Studio showed varying results, with the 31B model experiencing output issues. The author also experimented with the models through the llm-gemini API, generating SVG images of a pelican riding a bicycle to assess their visual capabilities.
The models support both vision and audio input, although audio functionality is not yet fully implemented in tools like LM Studio or Ollama. Testing with LM Studio showed varying results, with the 31B model experiencing output issues. The author also experimented with the models through the llm-gemini API, generating SVG images of a pelican riding a bicycle to assess their visual capabilities.
This document details how to run Google's Gemma 4 models locally, including the E2B, E4B, 26B-A4B, and 31B variants. Gemma 4 is a family of open models supporting over 140 languages and up to 256K context, available in both dense and MoE configurations. The E2B and E4B models support image and audio input. These models can be run locally on your device and fine-tuned using Unsloth Studio. The document outlines hardware requirements, recommended settings, and best practices for prompting and multimodal use, including guidance on context length and thinking mode.
