LLM 0.17 release enables multi-modal input, allowing users to send images, audio, and video files to Large Language Models like GPT-4o, Llama, and Gemini, with a Python API and cost-effective pricing.
Simon Willison explains how to use the mistral.rs library in Rust to run the Llama Vision model on a Mac M2 laptop. He provides a detailed example and discusses the memory usage and GPU utilization.
Simon Willison recently delivered a talk during the Mastering LLMs: A Conference For Developers & Data Scientists, which was a six-week long online event. The talk centered around Simon's LLM Python command-line utility and its plugins, emphasizing how they can be utilized to explore Large Language Models (LLMs) and perform various tasks. Last week, he discussed accessing LLMs from the command-line, sharing valuable insights and techniques with the audience.
* **http.server**: Run a localhost web server on port 8000: `python -m http.server`
* **base64**: Encode/decode base64: `python -m base64 -h`
* **asyncio**: Python console with top-level `await`: `python -m asyncio`
* **tokenize**: Debug mode for Python tokenizer: `python -m tokenize cgi.py`
* **ast**: Debug mode for Python AST module: `python -m ast cgi.py`
* **json.tool**: Pretty-print JSON: `echo '{"foo": "bar"}' | python -m json.tool`
* **random**: Benchmarking suite for random number generators (fixed in Python 3.13)
* **nntplib**: Display latest articles in a newsgroup: `python -m nntplib`
* **calendar**: Show a calendar for the current year: `python -m calendar` (with options like `-t html`)
* **base64**: Encode/decode base64: `python -m base64 -h`
* **asyncio**: Python console with top-level `await`: `python -m asyncio`
* **tokenize**: Debug mode for Python tokenizer: `python -m tokenize cgi.py`
* **ast**: Debug mode for Python AST module: `python -m ast cgi.py`
* **json.tool**: Pretty-print JSON: `echo '{"foo": "bar"}' | python -m json.tool`
* **random**: Benchmarking suite for random number generators (fixed in Python 3.13)
* **nntplib**: Display latest articles in a newsgroup: `python -m nntplib`
* **calendar**: Show a calendar for the current year: `python -m calendar` (with options like `-t html`)
Mixtral 8x7B:
Use llm-llama-cpp plugin.
Download a GGUF file for Mixtral 8X7B Instruct v0.1.
Run the model using llm -m gguf with the downloaded file.
Use llm-llama-cpp plugin.
Download a GGUF file for Mixtral 8X7B Instruct v0.1.
Run the model using llm -m gguf with the downloaded file.
The "LLM" toolkit provides a command-line utility and Python library for interacting with large language models. It enables users to run prompts from the terminal, store responses in SQLite databases, generate embeddings, and more. This comprehensive guide includes topics such as setup, usage, OpenAI models, other models, embeddings, plugins, model aliases, Python API, prompt templates, logging, related tools, CLI reference, contributing, and changelog.
* **New Feature:** `openai-to-sqlite` tool now allows enriching data in a SQLite database using OpenAI's GPT3.5 model.
* **Sentiment Analysis Example:**
+ Update a `sentiment` column in a `messages` table using `chatgpt()` function.
* **Sentiment Analysis Example:**
+ Update a `sentiment` column in a `messages` table using `chatgpt()` function.
The author has also automated their weeknotes by using an Observable notebook, which generates the "releases this week" and "TILs this week" sections.
The notebook fetches TILs from the author's Datasette, grabs releases from GitHub, and assembles a markdown string for the new post.
* `llm` CLI tool for running prompts against large language models
* Automation of weeknotes using an Observable notebook
* Notebook generates "releases this week" and "TILs this week" sections
* Tool stores prompts and responses in a SQLite database
The notebook fetches TILs from the author's Datasette, grabs releases from GitHub, and assembles a markdown string for the new post.
* `llm` CLI tool for running prompts against large language models
* Automation of weeknotes using an Observable notebook
* Notebook generates "releases this week" and "TILs this week" sections
* Tool stores prompts and responses in a SQLite database
