LLM Council works together to answer your hardest questions. A local web app that uses OpenRouter to send queries to multiple LLMs, have them review/rank each other's work, and finally a Chairman LLM produces the final response.
The article discusses the evolution of model inference techniques from 2017 to a projected 2025, highlighting the progression from simple frameworks like Flask and FastAPI to more advanced solutions like Triton Inference Server and vLLM. It details the increasing demands on inference infrastructure driven by larger and more complex models, and the need for optimization in areas like throughput, latency, and cost.
FastDeploy is a Bash script to automate FastAPI application deployment on Debian-based servers, using Systemd for service management and Nginx as a reverse proxy. It provides easy and advanced configuration modes, dependency management, user and permission management, and automated SSL setup.
This tutorial details how to use FastAPI-MCP to convert a FastAPI endpoint (fetching US National Park alerts) into an MCP-compatible server. It covers environment setup, app creation, testing, and MCP server implementation with Cursor IDE.
Docling is a powerful open-source library for document processing, supporting diverse formats and advanced PDF understanding, with seamless integrations with the gen AI ecosystem.
txtai is an open-source embeddings database for various applications such as semantic search, LLM orchestration, language model workflows, and more. It allows users to perform vector search with SQL, create embeddings for text, audio, images, and video, and run pipelines powered by language models for question-answering, transcription, translation, and more.
This is a hands-on guide with Python example code that walks through the deployment of an ML-based search API using a simple 3-step approach. The article provides a deployment strategy applicable to most machine learning solutions, and the example code is available on GitHub.
How to use Burr, an open source framework using simple OpenAI client calls to GPT4, and FastAPI to create a custom email assistant agent. We’ll describe the challenge one faces and then how you can solve for them.
