A benchmark of large language models, sorted by size (on disk) for each score. Highlighted entries are on the Pareto frontier.
Weaviate introduces StructuredRAG, a benchmark to evaluate LLMs' ability to generate reliable JSON outputs. The study finds that while LLMs perform well on simpler tasks, they struggle with more complex outputs.
This repository contains scripts for benchmarking the performance of large language models (LLMs) served using vLLM.
A startup called Backprop has demonstrated that a single Nvidia RTX 3090 GPU, released in 2020, can handle serving a modest large language model (LLM) like Llama 3.1 8B to over 100 concurrent users with acceptable throughput. This suggests that expensive enterprise GPUs may not be necessary for scaling LLMs to a few thousand users.
Independent analysis of AI language models and API providers. Understand the AI landscape and choose the best model and API provider for your use-case.
This article explores the concept of quantization in large language models (LLMs) and its benefits, including reducing memory usage and improving performance. It also discusses various quantization methods and their effects on model quality.
A Github Gist containing a Python script for text classification using the TxTail API
Compare the performance of different LLM that can be deployed locally on consumer hardware. The expected good response and scores are generated by GPT-4.
