Explicit criteria to cut false positives
Why vague review and classification instructions such as "be conservative" or "only report high-confidence findings" fail, and how categorical report/skip lists, severity definitions with code examples, evidence rules, REVIEW.md and a labeled evaluation set raise precision without hiding real issues.
Design prompts with explicit criteria to improve precision and reduce false positives
~8 min read · 9 practice questions ·
What the exam expects
- Replace a vague instruction ("check that comments are accurate", "flag anything unusual", "escalate urgent tickets") with a categorical criterion that says exactly which findings to report and which to skip.
- Explain why "be conservative", "only report high-confidence findings" and confidence-score thresholds fail to raise precision, and reject them as distractors.
- Restore developer trust by temporarily disabling a high false-positive category while its criteria are rewritten, keeping accurate categories running.
- Define severity levels with explicit criteria and a concrete code example per level so classification stays consistent across runs.
- Require evidence for a class of finding, tune managed Code Review with a committed
REVIEW.md, and prove a new prompt on a labeled set before rollout.
Key terms: precision (share of reported findings that are real); recall (share of real issues that get reported); false positive (a reported finding that is not a real problem); categorical criterion (a rule naming the issue types to report or skip, not a confidence level); REVIEW.md (repository-root file of review-only instructions for Code Review).
Key facts
- Detail
- Do not improve precision compared with specific categorical criteria
- Detail
- Temporarily disable it to restore trust while its prompt is improved
- Detail
- Explicit criteria plus a concrete code example per level
REVIEW.md location- Detail
- Repository root, read from the base or default branch, never the PR's changes
- Detail
- Severity lowering, exclusions and nit caps from the root file are honored
- Detail
- Require
file:lineevidence, not inference from naming
- Detail
- Specific and measurable; precision and recall are generic metrics
- Why does appending "be conservative" to "flag anything unusual" barely change the false positive rate?
- When should you turn off a whole review category?
- Why is mapping severity to Claude's confidence a poor design?
- How do you show a rewritten prompt cuts false positives without hiding real bugs?
The questions are free to read. Answers with their sources and self-grading are included with the pack.
Keep reading with the Claude Certified Architect - Foundations pack
- Core concepts
- Exam traps
- Worked scenario
The pack includes:
- All 30 lessons of the Claude Certified Architect - Foundations guide, with reading progress
- Interactive decision walks and step-by-step sequence steppers
- Inline practice questions and one-click topic practice (9 questions for this lesson)
- An official source cited for every claim
- An AI tutor that answers from the guide
- Unlimited timed practice exams
Practice for the Claude Certified Architect - Foundations exam
Certifyforce is an independent study resource and is not affiliated with, endorsed by or sponsored by Anthropic. Claude is a trademark of Anthropic.