A collection of Python examples demonstrating the use of Mistral.rs, a Rust library for working with mistral models.
Fast and easy LLM serving for the mac.
An exploration of advanced Rust programming concepts through a snippet of asynchronous code, illustrating the depth and complexity of real-world Rust development.
Simon Willison explains how to use the mistral.rs library in Rust to run the Llama Vision model on a Mac M2 laptop. He provides a detailed example and discusses the memory usage and GPU utilization.
The US Defense Advanced Research Projects Agency (DARPA) is developing TRACTOR, a project aimed at using AI to automatically convert legacy C code into Rust for improved memory safety.
uv is a drop-in replacement for common pip, pip-tools, and virtualenv commands, providing 10-100x faster installation and syncing compared to pip and pip-tools. It offers disk-space efficient global cache, advanced features, and best-in-class error messages. uv is backed by Astral, creators of Ruff.
Super Long Term Time-Lapse Camera and Monitoring by OpenAI. Autofocus camera designed for long-duration time-lapse photography with energy-efficient DeepSleep mode, Wi-Fi connectivity, and AI integrations.
Utilities for Llama.cpp, OpenAI, Anthropic, Mistral-rs. A collection of tools for interacting with various large language models. The code is written in Rust and includes functions for loading models, tokenization, prompting, text generation, and more.
Service Development Kit that uses Terraform, AWS ECS, Rust, Actix App, Postgress RDS, LLM, RAG, Cloudflare
• step-by-step guide on how to set up the service development kit, including creating an SSL certificate, setting up Terraform, and configuring Cloudflare.
• Rust, LLM, and RAG in the service development kit.
• step-by-step guide on how to set up the service development kit, including creating an SSL certificate, setting up Terraform, and configuring Cloudflare.
• Rust, LLM, and RAG in the service development kit.
Mistral.rs is a fast LLM inference platform supporting inference on a variety of devices, quantization, and easy-to-use application with an Open-AI API compatible HTTP server and Python bindings. It supports the latest Llama and Phi models, as well as X-LoRA and LoRA support. The project aims to provide the fastest LLM inference platform possible.
