Tags: quantization* + llm*

0 bookmark(s) - Sort by: Date ↓ / Title /

  1. Not Mixtral MoE but Merge-kit MoE

    • What makes a perfect MoE: The secret formula
    • Why is a proper merge considered a base model, and how do we distinguish them from a FrankenMoE?
    • Why the community working together to improve as a whole is the only way we will get Mixtral right
  2. Improving the memory and computational efficiency of Large Language Models (LLMs) for handling long input sequences, including retrieval augmented questions answering, summarization, and chat tasks. It covers various techniques, such as lower precision computing, Flash Attention algorithm, positional embedding methods, and key-value caching strategies. These methods help reduce memory consumption and increase inference speeds while maintaining high accuracy levels in LLM applications. Furthermore, it highlights some advanced approaches like Multi-Query-Attention (MQA) and Grouped-Query-Attention (GQA), which further enhance computational and memory efficiency without compromising performance.

  3. 2024-01-29 Tags: , , , , by klotz
  4. 2024-01-28 Tags: , , , , by klotz
  5. Exploring Pre-Quantized Large Language Models

    2023-11-15 Tags: , , by klotz
  6. 2023-06-09 Tags: , , , , , , by klotz
  7. 2023-06-06 Tags: , , , , by klotz

Top of the page

First / Previous / Next / Last / Page 2 of 0 SemanticScuttle - klotz.me: tagged with "quantization+llm"

About - Propulsed by SemanticScuttle