SemanticScuttle - klotz.me

klotz: streamingllm*

0 bookmark(s) - Sort by: Date ↓ / Title / - Bookmarks from other users for this tag

1. Add StreamingLLM for llamacpp & llamacpp_HF (2nd attempt)

   This pull request adds StreamingLLM support for llamacpp and llamacpp_HF models, aiming to improve performance and reliability. The changes allow indefinite chatting with the model without re-evaluating the prompt.

   2024-11-26 Tags: streamingllm, llamacpp, llm, chat, oobabooga by klotz
2. StreamingLLM (llama.cpp & llamacpp_HF loaders)

   This PR implements the StreamingLLM technique for model loaders, focusing on handling context length and optimizing chat generation speed.

   2024-11-26 Tags: streamingllm, llama.cpp, context, chat, llm, oobabooga by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0