petals.dev
Petals is a project for running large language models at home, BitTorrent-style. Users load part of a model and join a network of people serving its other parts. It supports generating text with Llama 3.1, Mixtral, Falcon and BLOOM and fine-tuning them on a consumer-grade GPU or Google Colab.