This article details building end-to-end observability for LLM applications using FastAPI and OpenTelemetry. It emphasizes a code-first approach, manually designing traces, spans, and semantic attributes to capture the full lifecycle of LLM-powered requests. The guide advocates for a structured approach to tracing RAG workflows, focusing on clear span boundaries, safe metadata capture (hashing prompts/responses), token usage tracking, and integration with observability backends like Jaeger, Grafana Tempo, or specialized LLM platforms. It highlights the importance of understanding LLM behavior beyond traditional infrastructure metrics.
This course takes you from Python fundamentals to AI Agent development, covering core Python, NumPy, Pandas, SQL, Flask, FastAPI, LLMs, and open-source models via HuggingFace.
LLM Council works together to answer your hardest questions. A local web app that uses OpenRouter to send queries to multiple LLMs, have them review/rank each other's work, and finally a Chairman LLM produces the final response.
The article discusses the evolution of model inference techniques from 2017 to a projected 2025, highlighting the progression from simple frameworks like Flask and FastAPI to more advanced solutions like Triton Inference Server and vLLM. It details the increasing demands on inference infrastructure driven by larger and more complex models, and the need for optimization in areas like throughput, latency, and cost.
FastDeploy is a Bash script to automate FastAPI application deployment on Debian-based servers, using Systemd for service management and Nginx as a reverse proxy. It provides easy and advanced configuration modes, dependency management, user and permission management, and automated SSL setup.
This tutorial details how to use FastAPI-MCP to convert a FastAPI endpoint (fetching US National Park alerts) into an MCP-compatible server. It covers environment setup, app creation, testing, and MCP server implementation with Cursor IDE.
Docling is a powerful open-source library for document processing, supporting diverse formats and advanced PDF understanding, with seamless integrations with the gen AI ecosystem.
txtai is an open-source embeddings database for various applications such as semantic search, LLM orchestration, language model workflows, and more. It allows users to perform vector search with SQL, create embeddings for text, audio, images, and video, and run pipelines powered by language models for question-answering, transcription, translation, and more.
This is a hands-on guide with Python example code that walks through the deployment of an ML-based search API using a simple 3-step approach. The article provides a deployment strategy applicable to most machine learning solutions, and the example code is available on GitHub.
How to use Burr, an open source framework using simple OpenAI client calls to GPT4, and FastAPI to create a custom email assistant agent. We’ll describe the challenge one faces and then how you can solve for them.
