SemanticScuttle - klotz.me

Tags: llama.cpp* + llm* + quantization*

0 bookmark(s) - Sort by: Date ↓ / Title /

1. Text Generation Web UI

   This document details how to run Qwen models locally using the Text Generation Web UI (oobabooga), covering installation, setup, and launching the web interface.

   2025-04-08 Tags: alibaba, qwen, text generation web ui, oobabooga, llm, inference, llama.cpp, transformers, quantization, python by klotz
2. TIL: Quantize and use Llama 3.1 with llama.cpp on a Mac

   A guide on how to download, convert, quantize, and use Llama 3.1 8B model with llama.cpp on a Mac.

   2024-09-28 Tags: llama.cpp, quantization, llm, howto by klotz
3. llama.cpp quant names

   An explanation of the quant names used in the llama.cpp implementation, as well as information on the different types of quant schemes available.

   2024-06-23 Tags: llama.cpp, quantization, llm by klotz
4. The Most Simple Way to Set Up ChatGPT Locally

   2024-01-18 Tags: llm, quantization, llama.cpp, self-hosted, tutorial by klotz
5. How is LLaMa.cpp possible?

   2023-06-06 Tags: llama, llm, llama.cpp, quantization, self-hosted by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0