SemanticScuttle - klotz.me

Tags: benchmarks* + inference*

0 bookmark(s) - Sort by: Date ↓ / Title /

1. whichllm

   An open-source command-line tool designed to identify the optimal local Large Language Model specifically suited for a user's existing or planned hardware. It automatically detects GPU, CPU, and RAM capacity to rank HuggingFace models using real performance benchmarks instead of relying on parameter size alone.
   
   * Hardware auto-detection for NVIDIA, AMD, Apple Silicon, and CPUs
   * Intelligent ranking based on benchmark evidence and recency awareness
   * Capability to simulate different GPUs for hardware upgrade planning
   * Support for GGUF, AWQ, and GPTQ model formats
   * Streamlined workflows including one-command chat sessions and Python code snippet generation

   2026-06-12 Tags: python, cli, gpu, inference, benchmarks, huggingface, llm, andyyyy64, whichllm by klotz
2. Qwen3.5 GGUF Benchmarks

   This article details benchmarks for Unsloth Dynamic GGUFs of the Qwen3.5 model, including analysis of perplexity, KL divergence, and MXFP4. It covers performance across different bit widths and quant types, highlighting the impact of Imatrix and the limitations of certain quantization approaches. Full benchmark data is also provided.

   2026-03-01 Tags: qwen3.5, gguf, benchmarks, quantization, perplexity, kl divergence, mxfp4, imatrix, llm, inference, dynamic quantization, unsloth by klotz
3. guide : running gpt-oss with llama.cpp · Discussion #15396

   A detailed guide for running the new gpt-oss models locally with the best performance using `llama.cpp`. The guide covers a wide range of hardware configurations and provides CLI argument explanations and benchmarks for Apple Silicon devices.

   2025-10-04 Tags: llama.cpp, gpt-oss, large language model, inference, apple silicon, benchmarks, performance, gguf by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0