Introduces proxy-tuning, a lightweight decoding-time algorithm that operates on top of black-box LMs to achieve the same end as direct tuning. The method tunes a smaller LM, then applies the difference between the predictions of the small tuned and untuned LMs to shift the original predictions of the larger untuned model in the direction of tuning, while retaining the benefits of larger-scale pretraining.
- GitHub repository for a tutorial series called "0 to LitGPT."
- Provides an overview of how to get started with LitGPT, which is an open-source implementation of GPT-3.
- Offers various resources such as codes, issues, pull requests, actions, security features, insights, and more related to the LitGPT project.
- Provides an overview of how to get started with LitGPT, which is an open-source implementation of GPT-3.
- Offers various resources such as codes, issues, pull requests, actions, security features, insights, and more related to the LitGPT project.
