Arsen Apostolov writes about the actual electrical cost of running local Large Language Models on a single NVIDIA RTX 3090 compared to hosted cloud APIs.
>"I measured the actual GPU electricity for eight local models on one RTX 3090 — and the cheapest wasn't the smallest, nor the priciest the biggest"
Cost of Generating 1 Million Tokens Locally
| MODEL | PARAMS (Billions) | MEAN SPEED (tok/s) | AVG GPU DRAW (W) | € / 1M OUTPUT TOKENS |
| :--- | :---: | :---: | :---: | :---: |
| **gemma3:1b** | 1B | 136 tok/s | 154 W | €0.060 |
| **Qwen3-Coder** | 30.5B | 130 tok/s | 233 W | €0.112 |
| **gemma4:26b** | 26B | 85 tok/s | 246 W | €0.139 |
| **Devstral** | 24B | 49 tok/s | 320 W | €0.321 |
| **gemma3:27b** | 27B | 36 tok/s | 283 W | €0.361 |
| **Seed-OSS** | 36B | 4.5 tok/s | 186 W | €0.946 |
| **GLM-4.5-Air** | 106B | 5.7 tok/s | 141 W | €1.040 |
| **DeepSeek-R1-Distill** | 32.8B | 6.9 tok/s | 155 W | €1.526 |
By measuring real-time GPU power consumption through a custom dashboard, he discovered that token costs are driven by effective wall-clock throughput rather than model parameter size or raw generation speed alone. The results show that while small and fast models can be more economical than cloud services, reasoning-heavy models may actually become the most expensive to run locally due to the time spent "deliberating" between tokens.
* Measurements were performed using HomeLab Monitor, an open-source dashboard that integrates live power data from `nvidia-smi`.
* DeepSeek-R1-Distill emerged as the most expensive model per million tokens because its effective throughput is slowed by reasoning delays.
* The findings focus on marginal electricity costs and exclude total cost of ownership factors like hardware amortization or idle draw.
>"I measured the actual GPU electricity for eight local models on one RTX 3090 — and the cheapest wasn't the smallest, nor the priciest the biggest"
Cost of Generating 1 Million Tokens Locally
| MODEL | PARAMS (Billions) | MEAN SPEED (tok/s) | AVG GPU DRAW (W) | € / 1M OUTPUT TOKENS |
| :--- | :---: | :---: | :---: | :---: |
| **gemma3:1b** | 1B | 136 tok/s | 154 W | €0.060 |
| **Qwen3-Coder** | 30.5B | 130 tok/s | 233 W | €0.112 |
| **gemma4:26b** | 26B | 85 tok/s | 246 W | €0.139 |
| **Devstral** | 24B | 49 tok/s | 320 W | €0.321 |
| **gemma3:27b** | 27B | 36 tok/s | 283 W | €0.361 |
| **Seed-OSS** | 36B | 4.5 tok/s | 186 W | €0.946 |
| **GLM-4.5-Air** | 106B | 5.7 tok/s | 141 W | €1.040 |
| **DeepSeek-R1-Distill** | 32.8B | 6.9 tok/s | 155 W | €1.526 |
By measuring real-time GPU power consumption through a custom dashboard, he discovered that token costs are driven by effective wall-clock throughput rather than model parameter size or raw generation speed alone. The results show that while small and fast models can be more economical than cloud services, reasoning-heavy models may actually become the most expensive to run locally due to the time spent "deliberating" between tokens.
* Measurements were performed using HomeLab Monitor, an open-source dashboard that integrates live power data from `nvidia-smi`.
* DeepSeek-R1-Distill emerged as the most expensive model per million tokens because its effective throughput is slowed by reasoning delays.
* The findings focus on marginal electricity costs and exclude total cost of ownership factors like hardware amortization or idle draw.
