SemanticScuttle - klotz.me

Tags: llama* + deep learning*

0 bookmark(s) - Sort by: Date ↓ / Title /

1. The Optimal Architecture for Small Language Models

   This article details research into finding the optimal architecture for small language models (70M parameters), exploring depth-width tradeoffs, comparing different architectures, and introducing Dhara-70M, a diffusion model offering 3.8x faster throughput with improved factuality.

   2025-12-27 Tags: llm, nlp, small language models, architecture, diffusion, llama, gemma, deep learning by klotz
2. Chess Llama - Training a tiny Llama model to play chess

   This blog post details the training of 'Chess Llama', a small Llama model designed to play chess. It covers the inspiration behind the project (Chess GPT), the dataset used (Lichess Elite database), the training process using Huggingface Transformers, and the model's performance (Elo rating of 1350-1400). It also includes links to try the model and view the source code.

   2025-07-21 Tags: chess, llama, llm, machine learning, artificial intelligence, deep learning, transformers, huggingface, chessgpt, uci, pgn by klotz
3. The Big LLM Architecture Comparison

   A detailed comparison of the architectures of recent large language models (LLMs) including DeepSeek-V3, OLMo 2, Gemma 3, Mistral Small 3.1, Llama 4, Qwen3, SmolLM3, and Kimi 2, focusing on key design choices and their impact on performance and efficiency.

   2025-07-19 Tags: llm, large language models, deep learning, ai, architecture, deepseek, olmo, gemma, mistral, llama, qwen, smollm, kimi, moe, attention, transformers by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0