ORIGINAL TECHNICAL PRACTICE
Validate model-generated structured data
An LLM returns JSON for a workflow. What validation should happen before downstream systems use it?
Written for independent practice. The model answer presents one approach; clarify constraints and explain alternatives.
A REASONED APPROACH
Model answer.
Parse and validate the output against a versioned schema, then check business and authorization constraints separately. Syntactically valid JSON can still contain unsupported identifiers, fabricated facts or prohibited actions.
Work through it
- Define required fields, types, ranges, enumerations and whether unknown fields are allowed. Reject invalid output before executing any side effect.
- Resolve referenced records through authorized application data and verify any evidence links. Use deterministic checks for totals, dates and state transitions where possible.
- Return a bounded repair request or a clear failure. Keep retries idempotent and cap their number so malformed output cannot create an endless workflow.
Tradeoffs to discuss
- A narrow schema improves reliability but may need explicit escape hatches for genuinely unsupported requests. Constrained generation helps shape the response without proving its factual correctness.
Common pitfalls
- Do not treat a model-generated approval flag as authorization. Silently coercing invalid values can convert an obvious error into a plausible but incorrect business action.