Ashish Vaswani et. al. introduce Transformers and Attention in this classic 2017 paper.
The Transformer architecture relies solely on attention mechanisms, dispensing with recurrence and convolutions entirely for sequence transduction tasks. This new network design improves translation quality while being more parallelizable and significantly faster to train than previous models.
- Achieved 28.4 BLEU on the WMT 2014 English-to-German translation task.
- Reached a state-of-the-art score of 41.8 BLEU for English-to-French using eight GPUs in only 3.5 days.
- Demonstrates successful application to English constituency parsing with both large and limited training data sets.
The Transformer architecture relies solely on attention mechanisms, dispensing with recurrence and convolutions entirely for sequence transduction tasks. This new network design improves translation quality while being more parallelizable and significantly faster to train than previous models.
- Achieved 28.4 BLEU on the WMT 2014 English-to-German translation task.
- Reached a state-of-the-art score of 41.8 BLEU for English-to-French using eight GPUs in only 3.5 days.
- Demonstrates successful application to English constituency parsing with both large and limited training data sets.
This article explores different chunking strategies for Retrieval-Augmented Generation (RAG) systems, comparing nine approaches using the agenticmemory library to improve retrieval accuracy and reduce hallucinations.
A post with pithy observations and clear conclusions from building complex LLM workflows, covering topics like prompt chaining, data structuring, model limitations, and fine-tuning strategies.
This article details the often overlooked cost of storing embeddings for RAG systems, and how quantization techniques (int8 and binary) can significantly reduce storage requirements and improve retrieval speed without substantial accuracy loss.
This Space demonstrates a simple method for embedding text using a LLM (Large Language Model) via the Hugging Face Inference API. It showcases how to convert text into numerical vector representations, useful for semantic search and similarity comparisons.
This tutorial demonstrates how to build a powerful document search engine using Hugging Face embeddings, Chroma DB, and Langchain for semantic search capabilities.
Qodo-Embed-1-1.5B is a state-of-the-art code embedding model designed for retrieval tasks in the software development domain. It supports multiple programming languages and is optimized for natural language-to-code and code-to-code retrieval, making it highly effective for applications such as code search and retrieval-augmented generation.
Qodo releases Qodo-Embed-1-1.5B, an open-source code embedding model that outperforms competitors from OpenAI and Salesforce, enhancing code search, retrieval, and understanding for enterprise development teams.
This article provides a comprehensive guide on the basics of BERT (Bidirectional Encoder Representations from Transformers) models. It covers the architecture, use cases, and practical implementations, helping readers understand how to leverage BERT for natural language processing tasks.
An explanation of the differences between encoder- and decoder-style large language model (LLM) architectures, including their roles in tasks such as classification, text generation, and translation.
