Editorial draft available to read. Factual and language review is pending. Examples are illustrative.
Decision and method
Group questions by intent and consequences of error: services, pricing, boundaries, comparison and actions. Include clear, incomplete and ambiguous wording. Define required facts, unsupported claims that must not appear and expected behavior for each case. Do not reduce evaluation to matching one model answer word for word when several correct explanations are possible.
Illustrative example
Illustrative example: three questions ask about timing, but one describes known scope, another omits integrations and a third demands an absolute guarantee. Evaluation expects approved guidance, clarification and acknowledgement of the limit respectively. A confident, fluent response can fail any of these criteria despite sounding convincing during an informal demonstration.
Acceptance check
Reserve questions not used while tuning instructions and review errors by category. Preserve source versions, configuration and outcomes. Accept the evaluation when it identifies meaningful failures reproducibly and improvements on routine questions do not worsen behavior around uncertainty, permissions or external actions that require a different standard of correctness.
Key takeaways
- Evaluate facts and expected behavior using varied questions and explicit acceptance criteria.