Validation-retry loops for extraction quality
How to catch the semantic errors strict schemas let through, feed specific validation errors back to Claude, recognise when a retry cannot help because the data is absent, and design schema fields and finding metadata that surface discrepancies and false-positive patterns.
Implement validation, retry, and feedback loops for extraction quality
~8 min read · 10 practice questions ·
What the exam expects
- Tell a schema syntax error (which strict tool use eliminates) from a semantic error (values that do not sum, a value in the wrong field) and put business-rule validation in application code.
- Build a retry-with-error-feedback step: a follow-up that carries the original document, the failed extraction and the specific validation errors, or a
tool_resultwithis_error: truethat names the fields and rules. - Predict when a retry will work (format mismatches, structural errors) and when it cannot (the information is absent or sits in a document that was never sent), and route the second case instead of retrying.
- Design self-correction fields such as
calculated_totalbesidestated_totaland aconflict_detectedboolean. - Add a
detected_patternfield to structured findings so dismissals can be analysed by pattern.
Key terms: syntax error (output that does not parse or match the schema); semantic error (schema-valid output whose values are wrong); retry with error feedback (re-asking with the specific errors attached); self-correction field (a schema field that lets a discrepancy be recorded instead of hidden); dismissal analysis (grouping rejected findings to find false-positive patterns).
Key facts
- Detail
- Semantic errors (values don't sum, wrong field) still need validation
- Detail
- Structured outputs are "always valid" and type safe
minimum, maximum, multipleOf unsupported- Detail
- Unsupported schema features return a 400 error
- Detail
- Strip from the wire schema, note in description, validate the response
- Detail
- Compare enum values case-insensitively
- Detail
tool_resultwithtool_use_idandis_error: true; instructive, not "failed"
- Detail
- Then apologizes to the user
- Detail
- Original document, failed extraction, specific validation errors
- Detail
- Draft, review against criteria, refine; each a separate API call
- Detail
subtypeissuccessandstructured_outputis present
- Detail
error_max_structured_output_retries
Sources for this section (5)
- Exam guideeverpath-course-content.s3-accelerate.amazonaws.com (opens in a new tab)
- Structured outputsplatform.claude.com (opens in a new tab)
- Handle tool callsplatform.claude.com (opens in a new tab)
- Prompting best practicesplatform.claude.com (opens in a new tab)
- Get structured output from agentscode.claude.com (opens in a new tab)
- Why can a pipeline drop its JSON-repair retries after moving to strict tool use but not its business-rule validator?
- When will a retry with error feedback fail no matter how specific the feedback is?
- What should a
tool_resultthat reports a failed validation contain? - Why add a
detected_patternfield to review findings?
The questions are free to read. Answers with their sources and self-grading are included with the pack.
Keep reading with the Claude Certified Architect - Foundations pack
- Core concepts
- Exam traps
- Worked scenario
The pack includes:
- All 30 lessons of the Claude Certified Architect - Foundations guide, with reading progress
- Interactive decision walks and step-by-step sequence steppers
- Inline practice questions and one-click topic practice (10 questions for this lesson)
- An official source cited for every claim
- An AI tutor that answers from the guide
- Unlimited timed practice exams
Practice for the Claude Certified Architect - Foundations exam
Certifyforce is an independent study resource and is not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic.