SemanticScuttle - klotz.me

klotz: training* + llm* + deep learning*

0 bookmark(s) - Sort by: Date ↓ / Title / - Bookmarks from other users for this tag

1. Grokfast: Accelerated Grokking by Amplifying Slow Gradients

   This paper presents a method to accelerate the grokking phenomenon, where a model's generalization improves with more training iterations after an initial overfitting stage. The authors propose a simple algorithmic modification to existing optimizers that filters out the fast-varying components of the gradients and amplifies the slow-varying components, thereby accelerating the grokking effect.

   2024-08-19 Tags: grokking, deep learning, optimization techniques, gradient filtering, llm, training, eric hartford by klotz
2. How to train your large language model: A new technique speeds up the process

   This article discusses the process of training a large language model (LLM) using reinforcement learning from human feedback (RLHF) and a new alternative method called Direct Preference Optimization (DPO). The article explains how these methods help align the LLM with human expectations and make it more efficient.

   2024-05-15 Tags: llm, reinforcement learning, human feedback, openai, chatgpt, rlhf, dpo, training by klotz
3. Mastering LLM Techniques: Training

   Delving into transformer networks

   2023-11-18 Tags: nvidia, llm, training, transformers, deep learning by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0