AirLLM is an open-source library that allows large language models to run on consumer hardware using layer-wise inference. By loading layers sequentially, it enables 70B parameter models to operate on as little as 4GB of VRAM. Optimized for research and batch processing, it features block-wise quantization for up to 3x faster performance on Linux and Apple Silicon.
do not use hub but do reboot between two successive installs
hub will provide power (even unpowered hub) for testing but not enough for stable operation, which must be negotiated.
eventually i stopped getting errors about pre-load etc from
sudo kextutil -tn /Library/Extensions/SiLabsUSBDriver.kext
hub will provide power (even unpowered hub) for testing but not enough for stable operation, which must be negotiated.
eventually i stopped getting errors about pre-load etc from
sudo kextutil -tn /Library/Extensions/SiLabsUSBDriver.kext
