Prediction Hooks¶

Hooks let you observe or shape every decision Laya makes, without forking it.

They are the extension seam for the things every real deployment needs: audit logging, PII redaction before inference, caching, metrics, confidence gating, routing overrides, and forwarding a decision to an external service. They are opt-in: with no hooks configured the behaviour of Agent, Router and ONNXAgent is unchanged.

They are not only for direct calls. Each LangChain and LangGraph runnable takes the same five per-call arguments, so a hook can be attached to one node in a graph rather than to the whole agent.

This folder is the full reference. Start here, then dive into the page you need:

Quick start¶

import laya

def log(ctx):
    print(ctx.model, ctx.results[0]["answers"], ctx.elapsed_ms)

agent = laya.load("convaiinnovations/laya", on_predict_end=log)
agent.system_one("I was charged twice.", {"urgent": {"type": "noul", "instructions": "Urgent?"}})

An object can implement any subset of the lifecycle events:

class Audit:
    def on_predict_start(self, ctx):
        print("start", ctx.run_id)

    def on_predict_end(self, ctx):
        print("end", ctx.run_id, ctx.usage, ctx.elapsed_ms)

    def on_error(self, ctx):
        print("failed", ctx.run_id, ctx.error)

laya.load("convaiinnovations/laya", hooks=[Audit()])

Hooks can also be added later or scoped to a block:

agent.add_hook(tracer)               # attach at runtime
with agent.hooks_installed(debug):   # installed for the block, removed on exit
    agent.system_one(state, questions)

See runtime registration. For a hook that should apply everywhere without threading it through every call, register it once with process-wide defaults:

from laya import hooks

hooks.set_default_hooks(hooks=[Tracer()])

The mental model¶

There are three ideas.

1. A hook is a callable or an object. A plain function is convenient for one event; an object is convenient for several. Only the object form goes to hooks= -- a plain callable there is refused at construction, because hooks= reads for the six lifecycle methods (on_predict_start, on_predict_end, on_route, on_load, on_evict, on_error) and a bare function has none of them. Pass a plain callable as on_predict_start= or on_predict_end=; those are the single-event parameters, so a callable is exactly what they take.

2. Every hook of one call shares one mutable PredictContext. It carries the states, questions, results, routing decision, model name, usage, timing and any error. A call can carry many states at once (predict_batch), so a hook that means to cover every decision has to iterate ctx.states and ctx.results; ctx.usage and ctx.elapsed_ms are totals for the call. Because the context is mutable, a hook can shape the call, not only watch it: redact the state, rewrite the questions, replace the result, or skip inference with a cached answer.

3. There are two scopes. Agent hooks wrap a forward pass; Router hooks wrap routing plus inference and can also see model lifecycle (on_route, on_load, on_evict). This mirrors the "run hooks" vs "agent hooks" split in other agent frameworks.

                             Router.predict(state, questions)
   ┌──────────────────────────────────────────────────────────────────────────┐
   │  route()                                                                 │
   │    ├─ detect language / workflow                                         │
   │    └─ on_route          ctx.decision  (a hook may replace it)            │
   │                                                                          │
   │  load(decision.model)                                                    │
   │    ├─ build checkpoint on first use ──► on_load    ctx.model, ctx.agent  │
   │    └─ evict LRU checkpoint ───────────► on_evict   ctx.model             │
   │                                                                          │
   │  on_predict_start       ctx.states, ctx.questions, ctx.decision          │
   │    │                                                                     │
   │    ├── ctx.skip(results)? ──► skip the forward pass                      │
   │    │                                                                     │
   │    └── Agent.system_one(...)  ──►  Agent-level hooks run here            │
   │           on_predict_start  ─►  forward  ─►  on_predict_end              │
   │                                                                          │
   │  result["routing"] = decision                                            │
   │  on_predict_end         ctx.results, ctx.usage, ctx.elapsed_ms           │
   └──────────────────────────────────────────────────────────────────────────┘
                 any failure on the way ──► on_error, then on_predict_end

Scope at a glance¶

laya.serve and the MCP server call Router.predict, so Router hooks fire for them automatically. Agent hooks fire whenever the Router runs an attached or built agent.

Compatibility¶

• No hooks configured means no behavioural change. The unset path is regression-tested.
• All hook parameters are keyword arguments with defaults, so existing calls keep working.
• laya/hooks.py is pure Python: import laya does not pull in torch because of it.
• Hooks are synchronous by default. An async def event can be wrapped in AsyncHook, or passed as a plain async callable, and it runs to completion for you.
• hooks_timeout bounds a slow hook so it cannot hang a served request.
• Keep hooks fast and non-blocking; see errors and patterns for the consequences on laya.serve.

See also¶

• examples/hooks/: runnable audit, redact, cache and metrics hooks.
• tests/test_hooks.py: the behaviour spec.
• tests/test_hooks_api.py: the API-stability guard.
• laya/hooks.py: the implementation.