NandhaKishorM writes about Laya, a non-autoregressive System 1 decision engine designed to perform typed decisions—such as choices, scores, and yes/no answers—over text in over 100 languages. Using a single forward pass with an integrated router that selects the appropriate checkpoint per request, it achieves high speed (e.g., ~33 ms on a T4 GPU) without the need for text generation or parsing.
- Features three specific checkpoints: English, Multilingual (for 100+ languages), and Typed Decisions.
- Supports multiple decision primitives: `choice` (labels/probabilities), `score` (ordinal rubrics), and `noul` (binary probability).
- Offers a "Fast Path" using TileLang GPU kernels for significant latency reduction on NVIDIA hardware.
- Includes an HTTP server implementation (`laya-serve`) that is Jev-compatible via the `/v1/systemone` protocol.
- Provides TypeScript support through `laya-ts`, allowing inference in Node.js and browser environments via ONNX Runtime.
- Features three specific checkpoints: English, Multilingual (for 100+ languages), and Typed Decisions.
- Supports multiple decision primitives: `choice` (labels/probabilities), `score` (ordinal rubrics), and `noul` (binary probability).
- Offers a "Fast Path" using TileLang GPU kernels for significant latency reduction on NVIDIA hardware.
- Includes an HTTP server implementation (`laya-serve`) that is Jev-compatible via the `/v1/systemone` protocol.
- Provides TypeScript support through `laya-ts`, allowing inference in Node.js and browser environments via ONNX Runtime.
Deepan Wadhwa writes about OpenDecision, a Python package that makes structured semantic decisions using a zero-shot NLI model (~400M ModernBERT-large) instead of a generative LLM. It exposes three typed primitives—Choice (select from a set), Noul (binary predicate), and Score (ordered rubric)—and serves them locally via FastAPI with a TypeSafe SDK-compatible endpoint.
- Inspired by TypeSafe's Jev "System One Model" announcement; the author previously built similar fraud-detection logic for a healthcare client.
- Choice uses two complementary NLI "compilers" and falls back to a third adjudication pass when they disagree, all on the same model.
- On TypeSafe-adapted benchmarks: 84.3% Choice accuracy, 85.0% Noul, 0.375 MAE on Score; on its own 125-case holdout: 86.4%.
- The returned "confidence" value is a concentration measure over the probability distribution, not a calibrated correctness estimate.
- Inspired by TypeSafe's Jev "System One Model" announcement; the author previously built similar fraud-detection logic for a healthcare client.
- Choice uses two complementary NLI "compilers" and falls back to a third adjudication pass when they disagree, all on the same model.
- On TypeSafe-adapted benchmarks: 84.3% Choice accuracy, 85.0% Noul, 0.375 MAE on Score; on its own 125-case holdout: 86.4%.
- The returned "confidence" value is a concentration measure over the probability distribution, not a calibrated correctness estimate.
