Resource-efficient LLMs and Multimodal Models
A useful survey of resource-efficient LLMs and multimodal foundations models.
Provides a comprehensive analysis and insights into ML efficiency research, including architectures, algorithms, and practical system designs and implementations.
A useful survey of resource-efficient LLMs and multimodal foundations models.
Provides a comprehensive analysis and insights into ML efficiency research, including architectures, algorithms, and practical system designs and implementations.
deploy and run LLM (large language models), including LLaMA, LLaMA2, Phi-2, Mixtral-MOE, and mamba-gpt, on the Raspberry Pi 5 8GB.
My TL;DR guide for self hosting on a Pi.
Get Pi
Install DietPi
Install Docker, Docker-Compose
browse the LinuxServer.io fleet
create docker-compose file with needed 'stuff' (letsencrypt, NextCloud, etc)
run.
A pi4 4gb should run NextCloud plus other "stuff" without an issue (Mine has a bunch of *arr's). As other's mentioned, storage might be an issue. My Pi's all point back to a NAS for file storage/serving.
Get Pi
Install DietPi
Install Docker, Docker-Compose
browse the LinuxServer.io fleet
create docker-compose file with needed 'stuff' (letsencrypt, NextCloud, etc)
run.
A pi4 4gb should run NextCloud plus other "stuff" without an issue (Mine has a bunch of *arr's). As other's mentioned, storage might be an issue. My Pi's all point back to a NAS for file storage/serving.
