Schema-driven decisions¶

Turn a JSON schema, or a pydantic model, into Laya questions, and get back typed values with calibrated confidence. This is the bridge that makes Laya a structured-output engine: you describe the shape you want, Laya answers it in one forward pass.

import laya

schema = {
    "type": "object",
    "properties": {
        "department": {"type": "string", "enum": ["billing", "support", "sales"],
                       "description": "Which team should handle this?"},
        "urgency": {"type": "integer", "minimum": 0, "maximum": 2},
        "needs_human": {"type": "boolean"},
    },
}

agent = laya.load("convaiinnovations/laya")
values = agent.decide("I was charged twice, refund me.", schema=schema)
# {"department": "billing", "urgency": 2, "needs_human": True}

With pydantic (install laya[structured]):

from typing import Literal
from pydantic import BaseModel

class Ticket(BaseModel):
    department: Literal["billing", "support", "sales"]
    urgency: Literal[0, 1, 2]
    needs_human: bool

ticket = agent.decide("I was charged twice, refund me.", schema=Ticket)

The supported subset¶

The top level must be an object with properties. Each property becomes one question.

Every row below is a real schema: tests/test_structured_docs.py compiles the first column and asserts the question the compiler actually produces, so this table cannot drift from the code. A cell is either a property schema on its own, a whole schema when the row needs its $defs (the field is named field), or a call to an entry point.

Literal[...] and Optional[...] are the pydantic spellings of the enum and anyOf rows: questions_from_pydantic renders them to those shapes and the same rows apply. An Enum field (str/int mixins and IntEnum included) is the $ref row, which needs the whole schema to show: pydantic puts the members in $defs and points the property at them, so Optional[Dept] is an anyOf around that $ref.

title is not read. pydantic puts one on every field of model_json_schema() whether you asked for it or not, and a per-property name cannot label the per-option choices a question is built from, so the wording lever is description — see How it maps internally below.

Projection is exact: an enum: [1, 2, 3] returns 2, not "2"; a bounded integer returns a level between minimum and maximum; a boolean is noul >= 0.5.

Rejections¶

A schema that cannot be answered from a fixed option set raises laya.structured.SchemaError (a ValueError) naming the exact path. Each row is executed too, with the field named name:

The entry points themselves reject these; the last row needs a whole schema, because a $ref cycle runs through its $defs:

Limits: MAX_PROPERTIES = 32, MAX_OPTIONS = 32, MAX_SCORE_LEVELS = 10.

The API¶

Pass exactly one of schema or questions. With questions, decide returns the raw answers instead of projecting. Extra keyword arguments are forwarded to predict, so hooks, model=, task= and the token budget all work:

router.decide(state, schema=Ticket, model="multilingual", hooks=[Metrics()])

Scoring many states¶

decide_batch is the throughput form: the schema is planned once and its questions run over every state through predict_batch, so states share forward passes instead of one per call. Results come back in input order, projected exactly as decide projects, and return_details=True gives one DecisionResult per state:

values = agent.decide_batch(ticket_texts, schema=Ticket)          # values[i] matches ticket_texts[i]
results = router.decide_batch(states, schema=Ticket, return_details=True, batch_size=64)

On a Router each state is still routed on its own, so one call can span checkpoints. Which keywords reach predict_batch depends on who reads them. The call-level ones — batch_size=, hooks=, sort_by_length= — are passed as keywords to predict_batch. The routing pins and the token budget — model=, task=, lang=, lang_guess=, max_len=, head_max_len=, the tuple laya/structured.py::_ROUTER_REQUEST_KEYS names — are read off every request by a Router, whose predict_batch declares none of them, so decide_batch copies them onto each request instead; that is why decide_batch(states, schema=..., model="multilingual") pins a checkpoint the way decide does. On an Agent-like runner the split is visible: its predict_batch declares batch_size=, hooks= and lang= but no model= and swallows nothing, so model= there is a TypeError, not a silent extra keyword. Agent, ONNXAgent and Router all have it; a runner without predict_batch raises TypeError rather than silently falling back to a loop — call decide per state there. Batching can shift borderline argmaxes the same way predict_batch does; the README records the measured speedups for both devices.

Confidence and probabilities¶

By default decide returns only the values. Pass return_details=True for a DecisionResult with per-field confidence, probabilities, the raw answers, and the usage and routing of the call:

result = agent.decide(state, schema=Ticket, return_details=True)
result.values["department"]            # "billing"
result.answer_confidence["department"] # 0.94  max(p): the quantity min_confidence gates on
result.confidence["department"]        # 0.79  1 - H(p)/log(k) on this 3-option `choice` field
result.probabilities["department"]     # {"billing": 0.94, "support": 0.06, "sales": 0.0}
result.usage                           # {"input_tokens": ..., "output_tokens": 0,
                                       #  "state_tokens": ..., "state_tokens_dropped": ...,
                                       #  "truncated": ..., "truncated_questions": [...]}
result.routing                         # the Router decision, when a Router answered

usage is the block predict() built, forwarded whole: input_tokens sums the state over one row per question and output_tokens is always 0 because nothing is generated, while the other four keys are the truncation report (#174) -- state_tokens is what the whole serialized state needs, state_tokens_dropped the most of it any one question's head gave up, and truncated / truncated_questions say which. A seventh key, options, is present only when the head budget left some question's options sharing a token span (#538); docs/http-api.md documents both this block and that field as response keys, and tests/test_structured_docs.py holds this page's list to the code that builds it.

confidence and answer_confidence are different quantities, and the names follow the definitions. answer_confidence is max(p), the probability mass on the answer being reported. It is what temperature scaling fits, what every calibration figure in this repository is computed on, and what min_confidence is compared against — which is the reason to gate on it rather than on confidence. confidence is not one formula. On a choice or a score it is normalized entropy, 1 - H(p) / log(k), whose scale moves with the number of options the question had. On a noul it is max(p_true, 1 - p_true), the probability of the side being reported, which over two options is max(p). tests/test_confidence.py pins a two-option distribution as 0.90 on a noul against 0.53 on an equivalent choice; 0.53 is what entropy gives at k = 2, so the 0.90 is the second formula and not the first at a smaller option count. The schema above mixes both — department is a choice, needs_human a noul — so one DecisionResult.confidence dict can hold two scales at once, which is why neither reading compares against a threshold you have not measured per type. A field that reported no usable answer_confidence maps to None, which is not the same as a reported 0.0.

Gate on the same quantity the gate uses:

if result.answer_confidence["department"] < 0.6:
    result.values["department"] = "human-review"

answer_confidence being the right number to filter on is not the same as it being a trustworthy probability. Reading it as "about c of the answers returned at c are correct" holds only after temperatures have been fitted and validated on held-out data for that checkpoint and question shape. The shipped checkpoints are over-confident as shipped and laya-multilingual ships with no fitted temperatures at all — see the README's Calibration and Honest limits sections, and the fine-tuning notebook for the fitting loop. Fit before you rely on the level; report it because it is the quantity the gate and the eval harness both use.

How it maps internally¶

• Enum and Literal become choice questions with the values as string labels; the label is mapped back to the original value on the way out, so integers stay integers.
• A bounded integer becomes a score question with one level per value; the returned value is minimum + argmax.
• A boolean becomes a noul question; the value is noul >= 0.5.
• A description becomes the question instructions, so a good description is what makes the decision accurate. This follows the same rule as the hooks guide: be explicit about what each option means.
• A local $ref (#/$defs/... or #/definitions/...) is replaced by the definition it names before the field is planned, so a pydantic Enum asks exactly the question the matching Literal asks and projects to the member's value, which answer_to_pydantic turns back into the member. The property's own keys sit on top and the definition's description is dropped: pydantic fills it from the enum's docstring (pydantic v1 with "An enumeration." when there is none), which describes the type rather than the question, so the wording is still the field's description. A ref to another document, a missing entry or a cycle raises, and a ref to a model is a nested object.
• A null branch is dropped before the field is planned, so Optional[X] asks exactly the question X asks. The field's key is simply absent from the values when there is no answer for it, which is what makes it safe to declare a field optional without changing what the model sees.

See also¶

• Prediction hooks: observe, shape, cache or gate the decisions this produces.
• Decision primitives: choice, score and noul in depth.
• LangChain and LangGraph: LayaDecision is this call as a runnable in a chain.