This article explains BERT, a language model designed to understand text rather than generate it. It discusses the transformer architecture BERT is based on and provides a step-by-step guide to building and training a BERT model for sentiment analysis.
This tutorial covers fine-tuning BERT for sentiment analysis using Hugging Face Transformers. Learn to prepare data, set up environment, train and evaluate the model, and make predictions.
Discusses the trends in Large Language Models (LLMs) architecture, including the rise of more GPU, more weights, more tokens, energy-efficient implementations, the role of LLM routers, and the need for better evaluation metrics, faster fine-tuning, and self-tuning.
This article explains the Long RoPE methodology used to expand the context lengths in LLMs without significant performance degradation. It discusses the importance of context length in LLMs and the limitations of previous positional encoding methods. The article then introduces Rotational Positional Encoding (RoPE) and its limitations, and explains how Long RoPE extends RoPE to larger contexts.
python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('I love you'))"
Exploring the architecture of OpenAI’s Generative Pre-trained Transformers.
Delving into transformer networks
