Convai Innovations presents Laya, a multilingual, non-autoregressive system 1 decision model designed to provide typed answers with mathematically calibrated probabilities in a single forward pass. Unlike generative models, it does not generate text, thereby eliminating hallucinations and the need for parsing. The framework includes an automated Router that detects language and script to dispatch tasks to the most efficient checkpoint (English or Multilingual) within approximately 35ms on GPU.
- It is trained using Reinforcement Learning with Calibrated Decisions (RLCD) to ensure honest probability reporting.
- Laya can support context lengths of up to 8,192 tokens in its multilingual version.
- The model family includes specialized checkpoints like `laya-typed-decisions` which achieves significantly higher accuracy through fine-tuning on specific workflows.
- Performance benchmarks show it is roughly 6–8× faster than TypeSafe Jev for single question latency on a T4 GPU.
- It is trained using Reinforcement Learning with Calibrated Decisions (RLCD) to ensure honest probability reporting.
- Laya can support context lengths of up to 8,192 tokens in its multilingual version.
- The model family includes specialized checkpoints like `laya-typed-decisions` which achieves significantly higher accuracy through fine-tuning on specific workflows.
- Performance benchmarks show it is roughly 6–8× faster than TypeSafe Jev for single question latency on a T4 GPU.
NandhaKishorM writes about Laya, a non-autoregressive System 1 decision engine designed to perform typed decisions—such as choices, scores, and yes/no answers—over text in over 100 languages. Using a single forward pass with an integrated router that selects the appropriate checkpoint per request, it achieves high speed (e.g., ~33 ms on a T4 GPU) without the need for text generation or parsing.
- Features three specific checkpoints: English, Multilingual (for 100+ languages), and Typed Decisions.
- Supports multiple decision primitives: `choice` (labels/probabilities), `score` (ordinal rubrics), and `noul` (binary probability).
- Offers a "Fast Path" using TileLang GPU kernels for significant latency reduction on NVIDIA hardware.
- Includes an HTTP server implementation (`laya-serve`) that is Jev-compatible via the `/v1/systemone` protocol.
- Provides TypeScript support through `laya-ts`, allowing inference in Node.js and browser environments via ONNX Runtime.
- Features three specific checkpoints: English, Multilingual (for 100+ languages), and Typed Decisions.
- Supports multiple decision primitives: `choice` (labels/probabilities), `score` (ordinal rubrics), and `noul` (binary probability).
- Offers a "Fast Path" using TileLang GPU kernels for significant latency reduction on NVIDIA hardware.
- Includes an HTTP server implementation (`laya-serve`) that is Jev-compatible via the `/v1/systemone` protocol.
- Provides TypeScript support through `laya-ts`, allowing inference in Node.js and browser environments via ONNX Runtime.
