Output Handling and Structured Response Gates: Keeping a Wrong Answer From Becoming a Wrong Action

The first production incident I watched cleanly involved an agent that did not hallucinate. The model returned a perfectly fluent sentence, then attached a tool call with the wrong customer ID. The downstream system trusted the shape of what arrived, fired the action, and refunded the wrong account. Nobody at the post-mortem could find a hallucination in the transcript. The text was fine. The action was wrong.

That is the line I have learned to draw in every harness review since. A wrong answer is embarrassing. A wrong action is expensive. The output gate is the cheapest harness component you can install for the failures it actually prevents, because it is the last place between a confident language model and a system that is about to do something on its behalf.

What the output gate actually does

The output gate is not a content filter. It is a small pipeline that runs after the model has produced a response and before anything downstream sees it. Four steps, in this order, every time.

The funnel narrows on purpose. Each step has authority to stop the request and return a structured failure instead of letting the next step see malformed input. The audit log is not optional, because a gate that fires without a record is a gate that cannot be defended in front of an incident review or a regulator (the same reason this principle anchors the wider guardrails and runtime controls picture).

Schema validation: the cheapest control with the biggest payoff

JSON Schema validation on the model output is the single highest-leverage thing most teams can ship this week. The pattern is unsexy and decisive. You declare the response shape (field names, types, required fields, enumerations), you ask the model for that shape, and you reject the response if it does not match. Modern vendor APIs make this trivial: OpenAI’s structured outputs enforce the schema at decoding time, so the model literally cannot emit a malformed response. Anthropic’s tool-use API takes the same posture. Google’s function calling and Mistral’s JSON mode are in the same family.

For the cases where the vendor API does not enforce schemas natively (older models, smaller open-source models, more elaborate cases), the discipline lives in a library. Instructor is the one I reach for most often because it leans on Pydantic models the rest of the codebase already trusts. Outlines and Guidance go further with token-level constrained generation. The choice is a taste question. The non-negotiable is that the validation runs and the result is binary: matched the schema, or did not.

What schema validation buys you is the death of an entire failure class. The “the model returned text where I expected JSON” class, the “extra fields that broke our parser” class, the “missing required field that the downstream system filled with null” class. These are not LLM problems in any deep sense; they are interface problems, and we already know how to solve interface problems. The mistake in 2026 is still treating the model’s output as text-by-default instead of structure-by-default.

Refusal as a first-class output type

The second gate is the one most teams underbuild because it requires deciding, in advance, what your application will refuse. The vendor’s content policy refuses what the vendor refuses. Your application has to refuse what your application refuses, which is a different list.

A refusal pattern, in the sense that matters here, has three parts. A trigger (a regex, a classifier, a downstream lookup that returns “out of scope”). A structured refusal output (the same JSON shape as a normal response, but with a refused: true field and a reason code). And a logged decision, because the post-mortem question is never “did the model refuse” but “did the system refuse correctly, on what evidence, at what point in the pipeline.”

Make the refusal a real output type, not a string. The downstream UI handles a refusal differently than a malformed schema, differently than a successful response, and differently than a tool call that was blocked. Four states, four code paths, four log records. The applications I see fail the audit are the ones that conflate “the model could not answer” with “the model declined to answer” with “the policy blocked the answer.” Those are three different events. Treat them as three.

Tool-call gating: where the agentic systems eat teams alive

The third gate is the one that the agent era has made unavoidable. The model wants to call a tool. The tool, if called, will write to a database, send an email, charge a card, deploy a build, or post on someone’s behalf. The model has produced parameters for the call. Those parameters must be validated against the user’s permissions and the tool’s contract before the tool is invoked, not after.

The OWASP framework calls this LLM02 (Insecure Output Handling) when the output reaches a downstream system, and LLM08 (Excessive Agency) when the tool itself does too much. Read both together. The defence is the same: a gate that treats tool parameters as untrusted input, even when the model that produced them is your most-trusted vendor’s most-recent model. The OWASP Top 10 for LLM Applications is the document the practitioner discipline has converged on, and its insecure-output-handling guidance is the cheapest external authority you can point your team at.

Concretely, the tool-arg gate checks three things. First, does the current user have permission to perform this action on these resources (not “is the model allowed to call this tool” but “can this user, in this session, do what the model is asking”). Second, do the parameters parse cleanly against the tool’s expected types and ranges. Third, is the action within the policy boundary for this tool in this context (the model can write SQL, but only to the read-only replica; the model can send email, but only to addresses on the user’s contact list). Each check is a small decision, logged with its result, before any side effect runs.

This is the gate that prevents the wrong-customer-refund incident I opened with. The model produced a perfectly schema-valid response. The tool-arg gate would have noticed that the customer ID it produced did not belong to the user whose session was active, and refused the call before the refund ran. Schema validation alone could not have caught it. Refusal patterns alone could not have caught it. Only the gate that knows the user’s permission boundary can.

How the four steps fit together

The order matters. Schema validation runs first because everything downstream assumes a parseable response. Refusal check runs second because there is no point validating tool args on a response the application has already decided not to act on. Tool-arg validation runs third because it is the most expensive check (it touches the auth system and possibly the tool’s own contract) and should not run unless the response is shaped right and not refused. The side effect runs last, and only last.

Two things make this gate hold up under load. Each step returns a structured result that the next step can inspect rather than re-derive. And every gate’s decision is logged with enough context that an incident reviewer can reconstruct why the system did or did not act. That is the same audit discipline drawn around the whole guardrail layer, applied at the exit gate.

How to start

Pick the model interaction that does the most damage if its output is wrong, and put a schema on it this week. Add a structured refusal type next, with three or four explicit reason codes that map to the failure modes you have actually seen. Then take your most consequential tool and put a permission check in front of it, even if the permission check is a hand-written if-statement to begin with. Make every gate write a record.

What I would not do is buy a guardrail framework before the first three controls are in. Frameworks accelerate teams that already know what their gates are. Teams that adopt a framework before deciding their own contract usually end up with a generic refusal policy that catches the wrong things and misses the right ones. The work is not the library. The work is the four small, deliberate, logged decisions between the model and the side effect.