NandhaKishorM writes about Laya, a non-autoregressive System 1 decision engine designed to perform typed decisions—such as choices, scores, and yes/no answers—over text in over 100 languages. Using a single forward pass with an integrated router that selects the appropriate checkpoint per request, it achieves high speed (e.g., ~33 ms on a T4 GPU) without the need for text generation or parsing.
- Features three specific checkpoints: English, Multilingual (for 100+ languages), and Typed Decisions.
- Supports multiple decision primitives: `choice` (labels/probabilities), `score` (ordinal rubrics), and `noul` (binary probability).
- Offers a "Fast Path" using TileLang GPU kernels for significant latency reduction on NVIDIA hardware.
- Includes an HTTP server implementation (`laya-serve`) that is Jev-compatible via the `/v1/systemone` protocol.
- Provides TypeScript support through `laya-ts`, allowing inference in Node.js and browser environments via ONNX Runtime.
- Features three specific checkpoints: English, Multilingual (for 100+ languages), and Typed Decisions.
- Supports multiple decision primitives: `choice` (labels/probabilities), `score` (ordinal rubrics), and `noul` (binary probability).
- Offers a "Fast Path" using TileLang GPU kernels for significant latency reduction on NVIDIA hardware.
- Includes an HTTP server implementation (`laya-serve`) that is Jev-compatible via the `/v1/systemone` protocol.
- Provides TypeScript support through `laya-ts`, allowing inference in Node.js and browser environments via ONNX Runtime.
