llamafile is a Mozilla.ai project that packages LLMs into single-file executables by combining llama.cpp with Cosmopolitan Libc, enabling local inference across most operating systems and CPU architectures with no installation required. The project also includes whisperfile (speech-to-text via whisper.cpp) and diffusionfile (image generation via stable-diffusion.cpp). Version 0.10.0 introduced a major build system overhaul to track upstream llama.cpp more closely, adding GPU support (CUDA, ROCm, Metal, Vulkan), an interactive TUI, multimodal input, tool calling, and thinking-mode support.
- The build system uses Cosmopolitan Libc (cosmocc 4.0.2) to produce truly portable binaries in a single file.
- The build system uses Cosmopolitan Libc (cosmocc 4.0.2) to produce truly portable binaries in a single file.
A step-by-step guide on building llamafiles from Llama 3.2 GGUFs, including scripting and Dockerization.
The author explores the use of Gemma 2 and Mozilla's llamafile on AWS Lambda for serverless AI inference
The workflow triggers on push events on the 'master', 'main', and 'fix' branches, and runs on Ubuntu. It installs Make, caches the Cosmocc toolchain, sets up Cosmocc and Ape Loader, builds the project, makes a specific Llamafile, executes the Llama CLI CPU, and more.
Mixtral 8x7B:
Use llm-llama-cpp plugin.
Download a GGUF file for Mixtral 8X7B Instruct v0.1.
Run the model using llm -m gguf with the downloaded file.
Use llm-llama-cpp plugin.
Download a GGUF file for Mixtral 8X7B Instruct v0.1.
Run the model using llm -m gguf with the downloaded file.
Use large language models embedded in single-file executables from the command line to perform tasks like renaming images based on their visual content
Llamafile lets you distribute and run LLMs with a single file
