This pull request adds StreamingLLM support for llamacpp and llamacpp_HF models, aiming to improve performance and reliability. The changes allow indefinite chatting with the model without re-evaluating the prompt.
This PR implements the StreamingLLM technique for model loaders, focusing on handling context length and optimizing chat generation speed.
This project provides Dockerised deployment of oobabooga's text-generation-webui with pre-built images for Nvidia GPU, AMD GPU, Intel Arc, and CPU-only inference. It supports various extensions and offers easy deployment and updates.
A benchmark of large language models, sorted by size (on disk) for each score. Highlighted entries are on the Pareto frontier.
A web search extension for Oobabooga's text-generation-webui (now with nougat) that allows for web search integration with the AI.
Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vectors, now in oobabooga text generation webui!
An extension for Oobabooga's Text-Generation Web UI that retrieves and adds web content to the context of prompts for more informative AI responses.
An extension for oobabooga/text-generation-webui that enables the LLM to search the web using DuckDuckGo
An extension that automatically unloads and reloads your model, freeing up VRAM for other programs.
Training PRO extension for oobabooga WebUI - recent dev version. Key features and changes from the main Training in WebUI include:
- Chunking: precise raw text slicer (PRTS) uses sentence splitting and making sure things are clean on all ends
- Overlapping chunking: this special overlapping will make additional overlap block based on logical rules
- Custom scheduler: FP_low_epoch_annealing keeps the LR constant for the first epoch and uses cosine for the rest
- Target selector: Normal LORA is q, v, and it should be used with (q k v o) or (q k v)
- DEMENTOR LEARNING (experimental) is an experimental chunking to train long-form text in low numbers of epochs
- Chunking: precise raw text slicer (PRTS) uses sentence splitting and making sure things are clean on all ends
- Overlapping chunking: this special overlapping will make additional overlap block based on logical rules
- Custom scheduler: FP_low_epoch_annealing keeps the LR constant for the first epoch and uses cosine for the rest
- Target selector: Normal LORA is q, v, and it should be used with (q k v o) or (q k v)
- DEMENTOR LEARNING (experimental) is an experimental chunking to train long-form text in low numbers of epochs
