How to run Gemma 3 effectively with our GGUFs on llama.cpp, Ollama, Open WebUI and how to fine-tune with Unsloth! This page details running Gemma 3 on various platforms, including phones, and fine-tuning it using Unsloth, addressing potential issues with float16 precision and providing optimal configuration settings.
This document details how to run and fine-tune Gemma 3 models (1B, 4B, 12B, and 27B) using Unsloth, covering setup with Ollama and llama.cpp, and addressing potential float16 precision issues. It also highlights Unsloth's unique ability to run Gemma 3 in float16 on machines like Colab notebooks with Tesla T4 GPUs.
Google releases Gemma 3, a new iteration of their Gemma family of models. It ranges from 1B to 27B parameters, supports up to 128k tokens, accepts images and text, and supports 140+ languages. This article details its technical enhancements (longer context, multimodality, multilinguality) and provides information on inference with Hugging Face transformers, on-device deployment, and evaluation.
