A benchmark of large language models, sorted by size (on disk) for each score. Highlighted entries are on the Pareto frontier.
Weaviate introduces StructuredRAG, a benchmark to evaluate LLMs' ability to generate reliable JSON outputs. The study finds that while LLMs perform well on simpler tasks, they struggle with more complex outputs.
This repository contains scripts for benchmarking the performance of large language models (LLMs) served using vLLM.
A startup called Backprop has demonstrated that a single Nvidia RTX 3090 GPU, released in 2020, can handle serving a modest large language model (LLM) like Llama 3.1 8B to over 100 concurrent users with acceptable throughput. This suggests that expensive enterprise GPUs may not be necessary for scaling LLMs to a few thousand users.
Independent analysis of AI language models and API providers. Understand the AI landscape and choose the best model and API provider for your use-case.
This article explores the concept of quantization in large language models (LLMs) and its benefits, including reducing memory usage and improving performance. It also discusses various quantization methods and their effects on model quality.
A Github Gist containing a Python script for text classification using the TxTail API
- provides list of 55 categorical encoders, explains how to use the code as a supplement to Category Encoders Python module.
- categorizes encoders into families, explains how to reuse code from the benchmark to include your encoder or dataset in comparison
- categorizes encoders into families, explains how to reuse code from the benchmark to include your encoder or dataset in comparison
