Semantic search and document parsing tools for the command line. A collection of high-performance CLI tools for document processing and semantic search, built with Rust for speed and reliability.
The article discusses using Large Language Model (LLM) embeddings as features in traditional machine learning models built with scikit-learn. It covers the process of generating embeddings from text data using models like Sentence Transformers, and how these embeddings can be combined with existing features to improve model performance. It details practical steps including loading data, creating embeddings, and integrating them into a scikit-learn pipeline for tasks like classification.
A summary of a workshop presented at PyCon US on building software with LLMs, covering setup, prompting, building tools (text-to-SQL, structured data extraction, semantic search/RAG), tool usage, and security considerations like prompt injection. It also discusses the current LLM landscape, including models from OpenAI, Gemini, Anthropic, and open-weight alternatives.
This article details how to automate embedding generation and updates in Postgres using Supabase Vector, Queues, Cron, and pg_net extension with Edge Functions, addressing the issues of drift, latency, and complexity found in traditional external embedding pipelines.
A simple project demonstrating Retrieval Augmented Generation (RAG) using SQLite, sqlite-vec, and OpenAI. It embeds text files, stores them in a SQLite database, and retrieves relevant documents using vector search. The project features lightweight single-file SQLite databases, vector search capabilities, and OpenAI integration for embeddings and chat responses.
Learn how to automate AI embedding creation using PostgreSQL with pgai Vectorizer. Streamline your AI workflow with simple SQL commands.
ntegration: PGAI Vectorizer integrates AI capabilities into PostgreSQL, enabling users to generate AI embeddings directly within the database.
Ease of Use: It simplifies the process of creating embeddings using a single SQL command, eliminating the need for multiple tools and complex pipelines.
Automatic Sync: Embeddings are automatically updated as data changes, ensuring that embeddings stay current without manual intervention.
Model Flexibility: Users can quickly switch between different AI models without reprocessing data.
Scalability: Optimizes search performance with vector indexes, making it suitable for large datasets.
Customization: Allows users to define chunking and formatting rules to tailor embeddings to their specific needs.
ntegration: PGAI Vectorizer integrates AI capabilities into PostgreSQL, enabling users to generate AI embeddings directly within the database.
Ease of Use: It simplifies the process of creating embeddings using a single SQL command, eliminating the need for multiple tools and complex pipelines.
Automatic Sync: Embeddings are automatically updated as data changes, ensuring that embeddings stay current without manual intervention.
Model Flexibility: Users can quickly switch between different AI models without reprocessing data.
Scalability: Optimizes search performance with vector indexes, making it suitable for large datasets.
Customization: Allows users to define chunking and formatting rules to tailor embeddings to their specific needs.
Foundational concepts, practical implementation of semantic search, and the workflow of RAG, highlighting its advantages and versatile applications.
The article provides a step-by-step guide to implementing a basic semantic search using TF-IDF and cosine similarity. This includes preprocessing steps, converting text to embeddings, and searching for relevant documents based on query similarity.
The article provides a step-by-step guide to implementing a basic semantic search using TF-IDF and cosine similarity. This includes preprocessing steps, converting text to embeddings, and searching for relevant documents based on query similarity.
An article discussing the use of embeddings in natural language processing, focusing on comparing open source and closed source embedding models for semantic search, including techniques like clustering and re-ranking.
The author explores semantic search using embeddings on U.S. Presidents, comparing four models: BGE, ST, Ada, and Large. The findings show that while embeddings capture interesting data, their limitations and inability to understand subtext and perform certain semantic tasks highlight their shallowness compared to full language models.
Sage is a tool that allows developers to chat with any codebase using two commands. It provides a functional chat interface for code, supports running locally or on the cloud, and has a modular design for swapping components.
