Basics
The whole idea first, then the terms the rest of this page uses.
1.1Whole idea
Model output often drives code: a booking, a database call, an API request. A schema defines what valid means. Validate each reply, re-ask with the specific errors, and stop after a bounded number of attempts.
Never passes unvalidated model output downstream: validate against a schema and re-ask with the exact errors.
1.2Key concepts
One idea per card, in plain words.
Defines fields, types, ranges and allowed values. In Python, a Pydantic model plays this role.
Sends the errors back, so the model can correct itself.
Provider features that constrain output to a schema. Types pass; business rules still need checks.
Limits cost and latency. Then fail visibly.
Question
What exactly you must answer, and what to settle before you start.
2.1Question
An assistant extracts booking intents as JSON and sometimes returns invalid fields. Build the guard: validate, re-ask with errors, cap the retries and fail safely.
Detailed answer
How it works, and how to build or explain it.
3.1Step by step
The mechanism, in the order you would explain it.
- Define the schema
Name required fields, allowed values and ranges in one place.
- Parse, then validate
Invalid JSON counts as an error too. Report every problem, not just the first.
- Re-ask with specifics
Quote the exact errors in the next prompt.
- Cap the attempts
Two retries are usually enough. Count them as a metric.
- Check business rules
A valid type may still book a closed clinic. Run domain checks after the schema.
3.2Code
Working code for the core of it.
Never trust model output
Treat a model's reply as untrusted input. Validate it, and re-ask with the exact errors.
1export type Field = { type: "string" | "number" | "boolean"; required?: boolean; oneOf?: string[]; min?: number; max?: number };2export type Schema = Record<string, Field>;3 4export function validate(value: unknown, schema: Schema): string[] {5 if (typeof value !== "object" || value === null) return ["expected a JSON object"];6 const obj = value as Record<string, unknown>;7 const errors: string[] = [];8 for (const [key, f] of Object.entries(schema)) {9 const v = obj[key];10 if (v === undefined) {11 if (f.required) errors.push(`${key} is required`);12 continue;13 }14 if (typeof v !== f.type) errors.push(`${key} must be a ${f.type}`);15 else if (f.oneOf && !f.oneOf.includes(v as string)) errors.push(`${key} must be one of ${f.oneOf.join(", ")}`);16 else if (typeof v === "number" && ((f.min !== undefined && v < f.min) || (f.max !== undefined && v > f.max))) errors.push(`${key} is out of range`);17 }18 return errors;19}20 21/** Validate, then re-ask with the exact errors. Never pass unvalidated output downstream. */22export async function guardedCall<T>(prompt: string, llm: (p: string) => Promise<string>, schema: Schema, maxRetries = 2) {23 let ask = prompt;24 let errors: string[] = [];25 for (let attempt = 1; attempt <= maxRetries + 1; attempt++) {26 const raw = await llm(ask);27 try {28 const parsed = JSON.parse(raw) as T;29 errors = validate(parsed, schema);30 if (errors.length === 0) return { ok: true as const, value: parsed, attempts: attempt };31 } catch {32 errors = ["reply was not valid JSON"];33 }34 ask = `${prompt}\n\nYour last reply failed validation:\n- ${errors.join("\n- ")}\nReply with corrected JSON only.`;35 }36 return { ok: false as const, errors, attempts: maxRetries + 1 };37}validate(parsed, schema)- Checks type, range and allowed values, including for tool arguments.
ask = `${prompt}...`- Sends the exact errors back, which fixes most replies in one retry.
ok: false- After the retry limit, return a failure. Never pass bad output downstream.
Tested by npm run test:modules.
Q&A
What interviewers ask next. Answer out loud, then open.
4.1Interview questions
Why not rely on JSON mode?
What should happen after the last retry?
What should you track?
Extra depth
Edge cases, counterpoints and what to read next.
5.1Edge cases and counterpoints
Includes the strongest argument against this approach.
Show detailsHide details
- Field-level repair
- Fixing one bad field needs less than regenerating the whole reply.
- Counterpoint
- Heavy schemas can push models to fill fields with invented values. Allow 'unknown' explicitly.