Sarvam AI is releasing Sarvam 30B and Sarvam 105B as open-source models, trained from scratch on large-scale, high-quality datasets. These models demonstrate strong reasoning, programming, and agentic capabilities, with optimizations for efficient deployment across various hardware. Sarvam 30B powers Samvaad, while Sarvam 105B powers Indus. The release includes details on the model architecture, training process, benchmark results, and inference optimizations. The models are available on AI Kosh and Hugging Face, and the article details their performance across benchmarks and in real-world applications like webpage generation, JEE problem solving, and conversational agents.
This article details benchmarks for Unsloth Dynamic GGUFs of the Qwen3.5 model, including analysis of perplexity, KL divergence, and MXFP4. It covers performance across different bit widths and quant types, highlighting the impact of Imatrix and the limitations of certain quantization approaches. Full benchmark data is also provided.
A detailed guide for running the new gpt-oss models locally with the best performance using `llama.cpp`. The guide covers a wide range of hardware configurations and provides CLI argument explanations and benchmarks for Apple Silicon devices.
An analysis of the recent paper 'The Leaderboard Illusion' which critiques the Chatbot Arena's LLM evaluation methodology, focusing on issues with private testing, unfair sampling, and potential gaming of the leaderboard. It also explores OpenRouter as a potential alternative ranking system.
This article discusses the limitations of Large Language Models (LLMs) in classification tasks, focusing on their lack of uncertainty and the need for more accurate performance metrics. New benchmarks and a metric named OMNIACCURACY have been introduced to assess LLMs' capabilities in both scenarios with and without correct labels.
