Evaluate whether AI produces reviewable, permission-aware assistance—not whether it writes a persuasive answer. NIST’s Generative AI Profile identifies confabulation risks, including invented logic or citations that can mislead users. NIST AI 600-1. (nvlpubs.nist.gov)
Use a desk-designed adversarial test pack containing correct evidence, obsolete evidence, conflicting documents, missing information and restricted records.
| Test | Proposed expected behaviour |
|---|---|
| Missing support | Identifies the gap rather than inventing a compliant conclusion |
| Wrong period or entity | Does not silently use irrelevant evidence |
| Restricted material | Does not reveal it through answers, snippets or citations |
| Malicious instructions inside an uploaded document | Treats them as document content, not authorised instructions |
| Conflicting sources | Surfaces the conflict and preserves references |
| Proposed record change | Uses the agreed approval mechanism before consequential writes |
| Model or feature update | Supports repeatable regression testing |
Measure material-error frequency, reviewer correction time, abstentions, source accuracy and permission-test outcomes. Compare accepted outputs against a human-reviewed reference set; do not score fluent prose as correctness.
Ask contractually about model providers, retention, training use, subprocessors, processing location, feature disablement, audit logs and usage charges.
A useful recent reference is NIST SP 1353’s initial public draft, published on 19 August 2026, which provides illustrative AI-assisted CSF analysis use cases and simulated organisational materials. It is a draft, and its examples are explicitly not prescriptive assessment or assurance methodologies. NIST draft page. (csrc.nist.gov)
Proposed buying gate: do not approve autonomous consequential changes until the organisation has separately accepted the authority model, rollback arrangements and testing evidence.
Sources cited on this page
- NIST AI 600-1 ↗nvlpubs.nist.gov
- NIST draft page ↗csrc.nist.gov