Tags: structured outputs*

0 bookmark(s) - Sort by: Date ↓ / Title /

  1. Diogo Almeida writes that TypeSafe AI is releasing Jev, its first System One Model—a new class of frontier model built for fast, structured decisions that software can consume directly. Unlike autoregressive language models that generate strings token by token, Jev outputs type-safe structured values with calibrated probabilities in a single parallel query, achieving frontier-level intelligence on decision tasks at roughly 40–200× lower latency and cost. The company's new training method, Reinforcement Learning for Calibrated Decisions (RLCD), optimizes for epistemically honest probability estimates rather than human preference or verifiable rewards, and the architecture is mathematically incapable of producing type errors or hallucinations.
    - Named after William Stanley Jevons, whose paradox predicted that efficiency gains would increase (not decrease) total demand; TypeSafe expects each order-of-magnitude cost drop to unlock orders of magnitude more use cases.
    - Workflow evals benchmark Jev against the average of GPT-6 Astra and Fable 5.1 as reference probabilities, claiming 193.6× speed and 444.6× cost advantages on production-shaped tasks.
    - The team demonstrated real-time intelligence with a Doom bot making 10 structured queries per second (~$7/hour) and a Wikiracing bot that outperforms LLMs at high-cardinality link selection.
    - Jev supports output cardinality up to 255; for higher-cardinality choices it falls back to a two-stage scoring system that scores independently then makes an explicit selection.
  2. * **Structured Outputs:** Uses grammar-constrained decoding (logit biasing/masking) to enforce strict JSON schema compliance during inference. Best for deterministic data transformation.
    * **Function Calling:** Utilizes instruction tuning to enable model reasoning over tool definitions. Best for agentic workflows and external state mutation.

    | Feature | Structured Outputs | Function Calling |
    | :--- | :--- | :--- |
    | **Mechanism** | Constrained decoding (Grammar/Regex) | Instruction-tuned intent detection |
    | **Reliability** | 100% Schema Compliance | Probabilistic (requires retry logic) |
    | **Primary Use Case** | ETL, Query Gen, Reasoning traces | API Triggers, RAG, Task Routing |
    | **Latency/Cost** | Low overhead; optimized decoding | Higher overhead due to tool-definition tokens |

    * **ETL & Extraction:** Use Structured Outputs to ensure downstream parsers never fail on malformed JSON.
    * **Agentic Loops:** Use Function Calling for multi-turn interactions where the model must decide *which* tool to invoke based on context.
    * **Hybrid Pattern (Controller/Formatter):** Deploy a "Function Calling" agent as the **Controller** to select tools, then pipe results through a "Structured Output" layer as the **Formatter** to ensure clean data ingestion into databases or UIs.

Top of the page

First / Previous / Next / Last / Page 1 of 0 SemanticScuttle - klotz.me: tagged with "structured outputs"

About - Propulsed by SemanticScuttle