0 bookmark(s) - Sort by: Date ↓ / Title /
A tutorial on using Qwen2.5–7B-Instruct for creating a local, open-source, multi-agentic RAG system.
The implementation described in the article focuses on creating a multi-agentic Retrieval-Augmented Generation (RAG) system using code agents and the Qwen2.5–7B-Instruct model. The system consists of three agents working together in a hierarchical structure:
Manager Agent: This top-level agent breaks down user questions into sub-tasks, utilizes the Wikipedia search agent to find information, and combines the results to provide a final answer. Its system prompt is tailored to guide it through the process of decomposing tasks and coordinating with other agents.
Wikipedia Search Agent: This agent interacts with the Wikipedia search tool to identify relevant pages and their summaries. It further delegates to the page search agent for detailed information retrieval from specific pages if needed. Its prompt is designed to help it navigate Wikipedia effectively and extract necessary information.
Page Search Agent: This agent specializes in extracting precise information from a given Wikipedia page. It uses a semantic search tool to locate specific passages related to the query.
To implement the multi-agent system efficiently, the article mentions several key decisions and modifications to the default Hugging Face implementation:
By structuring the implementation with these considerations, the system achieves the capability to perform complex, multi-hop question-answering tasks efficiently, despite being powered by a relatively small model running on consumer-grade hardware
SmolVLM is a compact, efficient multimodal model designed for tasks involving text and image inputs, producing text outputs. It is capable of answering questions about images, describing visual content, and functioning as a pure language model without visual inputs. Developed for on-device applications, SmolVLM is lightweight yet performs well in multimodal tasks.
A comparison of frameworks, models, and costs for deploying Llama models locally and privately.
Microsoft has released the OmniParser model on HuggingFace, a vision-based tool designed to parse UI screenshots into structured elements, enhancing intelligent GUI automation across platforms without relying on additional contextual data.
Ollama now supports HuggingFace GGUF models, making it easier for users to run AI models locally without internet. The GGUF format allows for the use of AI models on modest-sized consumer hardware.
This paper analyzes the performance of 20 large language models (LLMs) using two inference libraries: vLLM and HuggingFace Pipelines. The study investigates how hyperparameters influence inference performance and reveals that throughput landscapes are irregular, highlighting the importance of hyperparameter optimization.
HuggingFace has released FineWeb, a new large-scale dataset consisting of 15 trillion tokens and 44TB of disk space designed for pretraining large language models (LLMs). The dataset, which leverages data from CommonCrawl, undergoes rigorous deduplication and quality filtering processes, making it a valuable tool for researchers.
This article explains how to use the Sentence Transformers library to finetune and train embedding models for a variety of applications, such as retrieval augmented generation, semantic search, and semantic textual similarity. It covers the training components, dataset format, loss function, training arguments, evaluators, and trainer.
This model was built using a new Smaug recipe for improving performance on real world multi-turn conversations applied to meta-llama/Meta-Llama-3-70B-Instruct.
The model outperforms Llama-3-70B-Instruct substantially, and is on par with GPT-4-Turbo, on MT-Bench (see below).
First / Previous / Next / Last / Page 1 of 0