SemanticScuttle - klotz.me

klotz: sglang*

0 bookmark(s) - Sort by: Date ↓ / Title / - Bookmarks from other users for this tag

1. SGLang - Home

   SGLang is a fast serving framework for large language models and vision language models. It focuses on efficient serving and controllable interaction through co-designed backend runtime and frontend language.

   2025-04-30 Tags: llm, vision language models, inference engineering, quantization, sglang by klotz
2. Achieving Faster Open-Source Llama3 Serving with SGLang Runtime (vs. TensorRT-LLM, vLLM)

   This blog post benchmarks and compares the performance of SGLang, TensorRT-LLM, and vLLM for serving large language models (LLMs). SGLang demonstrates superior or competitive performance in offline and online scenarios, often outperforming vLLM and matching or exceeding TensorRT-LLM.

   2024-07-27 Tags: sglang, tensorrt-llm, vllm, llama, llm by klotz

Top of the page

First / Previous / Next / Last / Page 1 of 0