- Discusses the use of consumer graphics cards for fine-tuning large language models (LLMs)
- Compares consumer graphics cards, such as NVIDIA GeForce RTX Series GPUs, to data center and cloud computing GPUs
- Highlights the differences in GPU memory and price between consumer and data center GPUs
- Shares the author's experience using a GeForce 3090 RTX card with 24GB of GPU memory for fine-tuning LLMs
GeForce 210 October 12, 2009 GT218 40 260 57 PCIe 2.0 x16
PCIe x1
PCI 16:8:4 520
589 1230
1402 1000–1600 2.356 4.712 512
1024 4.0
8.0
12.8 DDR2
DDR3 32
64 10.1 3.3 67.296 30.5
This is why cuda-12 doesn't work with podman 3.4.4 on ubuntu 22.04 I think:
- Rootless configuration for nvidia container runtime
- Setup missing hook for nvidia container runtime
- Increase memlock and stack ulimits
How to get oobabooga/text-generation-webui running on Windows or Linux with LLaMa-30b 4bit mode via GPTQ-for-LLaMa on an RTX 3090 start to finish.
Bookmarks on this page are managed by an admin user. 