SemanticScuttle - klotz.me

klotz: moe* + attention*

0 bookmark(s) - Sort by: Date ↓ / Title / - Bookmarks from other users for this tag

1. Inside GPT-OSS: OpenAI’s Latest LLM Architecture

   An in-depth look at the architecture of OpenAI's GPT-OSS models, detailing tokenization, embeddings, transformer blocks, Mixture of Experts, attention mechanisms (GQA and RoPE), and quantization techniques.

   2025-09-27 Tags: llm, gpt-oss, openai, transformer, mixture of experts, moe, attention, gqa, rope, quantization, machine learning, .qwen3–30b-a3b. by klotz
2. The Big LLM Architecture Comparison

   A detailed comparison of the architectures of recent large language models (LLMs) including DeepSeek-V3, OLMo 2, Gemma 3, Mistral Small 3.1, Llama 4, Qwen3, SmolLM3, and Kimi 2, focusing on key design choices and their impact on performance and efficiency.

   2025-07-19 Tags: llm, large language models, deep learning, ai, architecture, deepseek, olmo, gemma, mistral, llama, qwen, smollm, kimi, moe, attention, transformers by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0