A good answer should distinguish permission to read, recommend, approve and execute. Ask how the platform contains a mistaken or manipulated instruction.
Why it matters to buyers. NIST’s 2025 adversarial machine-learning taxonomy addresses attacks that manipulate AI inputs or interactions and can expose sensitive information. It supports examining adversarial behaviour in addition to ordinary output accuracy. (nvlpubs.nist.gov)
Put it to: Apostol Vassilev and the buyer’s security architect; include the supplier engineer responsible for the feature.
A strong answer should cover: - The identity and delegated authority under which an action runs. - Authorisation enforced outside model-generated instructions. - Restricted records, search results, citations, exports and downstream tool access. - Approval boundaries and separation between proposing and executing a material change. - Audit records, revocation, incident response and recovery of interrupted workflows. - Data retention, model-provider access and contractual treatment of buyer information.
Ask for proof: use synthetic restricted evidence and an authorised account with limited rights. Include a document containing instructions that attempt to redirect the AI, then revoke access and retest retrieval and action paths.
Set a proposed stop condition for observed unauthorised disclosure or action, while making clear that passing a finite test set is not proof that all such failures are impossible.
Follow-up: “If we disable the AI feature now, what queued actions, delegated credentials or external side effects remain?”
Expert quote — pending interview: [Insert approved quotation and attribution.]
Sources cited on this page
- nvlpubs.nist.gov ↗nvlpubs.nist.gov