A ruby script calculates VRAM requirements for large language models (LLMs) based on model, bits per weight, and context length. It can determine required VRAM, maximum context length, or best bpw given available VRAM.
Pinboard is a command-line utility for managing file references during raw language model development. It aids in streamlining codebase workflows, offering efficient context-aware file updates.
Noema Research introduces Pinboard, a developer tool for improved productivity. Pinboard, a command-line tool, efficiently manages files and terminal references, enhancing development workflows. Key features include flexible pinning, contextual updates, clipboard integration, an interactive shell, and undo functionality.
Simon Willison recently delivered a talk during the Mastering LLMs: A Conference For Developers & Data Scientists, which was a six-week long online event. The talk centered around Simon's LLM Python command-line utility and its plugins, emphasizing how they can be utilized to explore Large Language Models (LLMs) and perform various tasks. Last week, he discussed accessing LLMs from the command-line, sharing valuable insights and techniques with the audience.
A CLI tool for interacting with local or remote LLMs to retrieve information about files, execute queries, and perform other tasks in a Retrieval-Augmented Generation (RAG) fashion.
Retrochat is chat application that supports Llama.cpp, Kobold.cpp, and Ollama. It highlights new features, commands for configuration, chat management, and models, and provides a download link for the release.
The workflow triggers on push events on the 'master', 'main', and 'fix' branches, and runs on Ubuntu. It installs Make, caches the Cosmocc toolchain, sets up Cosmocc and Ape Loader, builds the project, makes a specific Llamafile, executes the Llama CLI CPU, and more.
Anthropic has introduced a new feature in their Console that allows users to generate production-ready prompt templates using AI. This feature employs prompt engineering techniques such as chain-of-thought reasoning, role setting, and clear variable delineation to create effective and precise prompts. It helps both new and experienced prompt engineers save time and often produces better results than hand-written prompts. The generated prompts are also editable for optimal performance.
Elia is a new open-source application designed for users who need an efficient, terminal-based way to interact with large language models (LLMs). The application allows users to chat with various models directly from their terminal and supports popular models like ChatGPT and Llama 3. Elia is highly keyboard-centric, stores conversations in a local SQLite database, and offers a simple setup via pipx.
The article argues that instead of developing numerous tools for LLM, giving it direct access to a terminal is more efficient and future-proof. It references Rich Sutton's "The Bitter Lesson" and discusses how the terminal's existing command-line tools can be utilized by LLM for various tasks, highlighting the importance of general methods over specialized tools.
