SemanticScuttle - klotz.me

Tags: self-hosted* + llama*

0 bookmark(s) - Sort by: Date ↓ / Title /

1. Deploy and run LLM on Raspberry Pi 5 vs Raspberry Pi 4B (LLaMA, LLaMA2, Phi-2, Mixtral-MOE, mamba-gpt) - DFRobot

   deploy and run LLM (large language models), including LLaMA, LLaMA2, Phi-2, Mixtral-MOE, and mamba-gpt, on the Raspberry Pi 5 8GB.

   2024-01-10 Tags: llm, rpi5, rpi, llama, mistral, self-hosted, dfrobot by klotz
2. Collection thread for llava accuracy : r/LocalLLaMA

   2023-12-04 Tags: llava, self-hosted, llama, llm, vision, reddit by klotz
3. jlonge4/local_llama: This repo is to showcase how you can run a model locally and offline, free of OpenAI dependencies.

   2023-06-25 Tags: llama.cpp, llama, pdf, chat, self-hosted, github, jlonge, hacks by klotz
4. text-generation-webui/presets at main · oobabooga/text-generation-webui

   2023-06-12 Tags: llama, presets, chatgpt, self-hosted, localllama by klotz
5. (2) How to install LLaMA: 8-bit and 4-bit : LocalLLaMA

   2023-06-12 Tags: llama, self-hosted, localllama by klotz
6. How is LLaMa.cpp possible?

   2023-06-06 Tags: llama, llm, llama.cpp, quantization, self-hosted by klotz
7. How to create a private ChatGPT that interacts with your local documents - TechTalks

   2023-06-05 Tags: llm, chatgpt, self-hosted, llama by klotz
8. Llama.Cpp

   # obtain the original LLaMA model weights and place them in ./models
   ls ./models
   65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model
   
   # install Python dependencies
   python3 -m pip install -r requirements.txt
   
   # convert the 7B model to ggml FP16 format
   python3 convert.py models/7B/
   
   # quantize the model to 4-bits (using q4_0 method)
   ./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin q4_0
   
   # run the inference
   ./main -m ./models/7B/ggml-model-q4_0.bin -n 128

   2023-06-05 Tags: github, llama, llama cpp, llm, self-hosted by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0