A collection of Python examples demonstrating the use of Mistral.rs, a Rust library for working with mistral models.
Fast and easy LLM serving for the mac.
Simon Willison explains how to use the mistral.rs library in Rust to run the Llama Vision model on a Mac M2 laptop. He provides a detailed example and discusses the memory usage and GPU utilization.
The US Defense Advanced Research Projects Agency (DARPA) is developing TRACTOR, a project aimed at using AI to automatically convert legacy C code into Rust for improved memory safety.
Utilities for Llama.cpp, OpenAI, Anthropic, Mistral-rs. A collection of tools for interacting with various large language models. The code is written in Rust and includes functions for loading models, tokenization, prompting, text generation, and more.
Service Development Kit that uses Terraform, AWS ECS, Rust, Actix App, Postgress RDS, LLM, RAG, Cloudflare
• step-by-step guide on how to set up the service development kit, including creating an SSL certificate, setting up Terraform, and configuring Cloudflare.
• Rust, LLM, and RAG in the service development kit.
• step-by-step guide on how to set up the service development kit, including creating an SSL certificate, setting up Terraform, and configuring Cloudflare.
• Rust, LLM, and RAG in the service development kit.
Mistral.rs is a fast LLM inference platform supporting inference on a variety of devices, quantization, and easy-to-use application with an Open-AI API compatible HTTP server and Python bindings. It supports the latest Llama and Phi models, as well as X-LoRA and LoRA support. The project aims to provide the fastest LLM inference platform possible.
It all started as a joke. I was in a group chat with a few of my friends and we were talking about football (soccer for the American readers). I entered the chat during a mildly heated discussion about the manager of a team one of my friends supports. It was going on for a bit while with seemingly no end in sight...
