This document details how to run and fine-tune Gemma 3 models (1B, 4B, 12B, and 27B) using Unsloth, covering setup with Ollama and llama.cpp, and addressing potential float16 precision issues. It also highlights Unsloth's unique ability to run Gemma 3 in float16 on machines like Colab notebooks with Tesla T4 GPUs.
This document details how to run Qwen models locally using the Text Generation Web UI (oobabooga), covering installation, setup, and launching the web interface.
Google releases Gemma 3, a new iteration of their Gemma family of models. It ranges from 1B to 27B parameters, supports up to 128k tokens, accepts images and text, and supports 140+ languages. This article details its technical enhancements (longer context, multimodality, multilinguality) and provides information on inference with Hugging Face transformers, on-device deployment, and evaluation.
Unify your existing devices into one powerful GPU: iPhone, iPad, Android, Mac, NVIDIA, Raspberry Pi, pretty much any device!
A USB stick equipped with a Raspberry Pi Zero W runs a large language model using llama.cpp. The project involves porting the model to an ARMv6 architecture and setting up the device as a composite that presents a filesystem to the host, allowing users to interact with the LLM by creating text files that are automatically filled with generated content.
llama.cppで使えるllama-cliコマンドのヘルプの翻訳です。 このコマンドをインストールするにはHomebrewを使用します。
This PR implements the StreamingLLM technique for model loaders, focusing on handling context length and optimizing chat generation speed.
A collection of lightweight AI-powered tools built with LLaMA.cpp and small language models.
A guide on how to download, convert, quantize, and use Llama 3.1 8B model with llama.cpp on a Mac.
A step-by-step guide on building llamafiles from Llama 3.2 GGUFs, including scripting and Dockerization.
