# 6. Where should evaluators probe beyond the standard demonstration?

### Can migration preserve the meaning of historical evidence?

**Require a preservation design, not just a successful import count.** Separate current operational data, historical versions, original provenance and records retained in a searchable archive.

A concrete documentation example shows why this matters: IBM’s OpenPages 9.1.x FastMap documentation states that FastMap cannot import attachments or signatures and cannot preserve original system creation dates through that process. These are limitations of the specified import mechanism—not proof that OpenPages has no alternative migration route. ([ibm.com](https://www.ibm.com/docs/en/openpages/9.1.x?topic=fastmap-overview&utm_source=openai))

Require the migration proposal to identify:

- The supported route for each object, attachment and history type.
- Source and target identifiers, transformation rules and relationship crosswalks.
- Whether original timestamps remain as source metadata, native history or archived evidence.
- Record, attachment and relationship reconciliation separately.
- Treatment of rejected records and duplicates.
- Freeze, delta-load, rollback and archive-access arrangements.

Do not silently represent imported source history as events generated natively by the new platform. Make provenance visible and have the records owner approve the treatment.

### What does a connector actually include?

**Score the integration contract and operating behaviour, not the connector name.** Require an interface schedule covering objects, direction, frequency, authentication, permitted actions, volumes, failure handling and maintenance ownership.

LogicGate’s published API getting-started page states that API access requires an add-on and discusses pagination and a recommended request rate. That is a useful prompt to verify commercial entitlement and throughput in the buyer’s actual offer; it is not a current quotation or a measured performance result. ([logicgate.ai](https://www.logicgate.ai/developer/risk-cloud-api-getting-started/))

For each critical interface, distinguish:

1. **Source validity:** does the incoming information represent the intended population and period?
2. **Transport reliability:** was it transferred completely and once, with detectable failures?
3. **GRC interpretation:** did the platform create the right evidence, finding or action?
4. **Operational ownership:** who notices, diagnoses and repairs a failure?

Include source-system and middleware charges in the commercial schedule where applicable. Make the supplier identify whether changes to the source API are covered by support or require paid engineering.

### Can occasional contributors do their work under the quoted licence?

**Map actions to entitlements, not only people to job titles.** A contributor test should use the proposed licence and permission combination.

For example, ServiceNow’s IRM entitlement document effective 9 May 2024 differentiates Operator and Lite Operator rights, including limits around internal-audit data. Treat that dated document as evidence that contractual distinctions matter—not as a substitute for the entitlement schedule governing a 2026 purchase. ([servicenow.com](https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/other-document/entitlements/PROD14209.pdf))

Ask the supplier to price and validate these actions separately: submitting evidence, approving an assessment, closing an issue, viewing reports, administering workflows and participating as an external auditor.

### Who operates the platform after the consultants leave?

**Require a workload model and a practical handover test.** Specify responsibility for access, configuration, data quality, integrations, releases, reporting, training and incident escalation.

A First Horizon employer posting dated 9 April 2026 for a Senior GRC Administrator explicitly includes configuration governance, data integrity, release control, integration validation and production stability. This is a concrete example of operational responsibilities—not evidence of a universal staffing requirement or salary benchmark. ([recruiting.ultipro.com](https://recruiting.ultipro.com/FIR1007FTN/JobBoard/c005ef3e-175a-49c4-ba29-9f431f673944/OpportunityDetail?opportunityId=3d3259bf-0bb5-474b-8eb4-639fe8f63a6b&utm_source=openai))

For the buyer’s operating model, estimate task volumes and effort rather than adopting an unsupported “one administrator” assumption. Record which responsibilities are internal, managed-service or supplier obligations and price them accordingly.

### How should AI features be evaluated?

**Make AI a small, separately governed part of the score unless it is genuinely central to the approved use case.** Do not award extra points merely for the number of AI-labelled features.

In its May 2026 GRC market commentary, Verdantix described expanding AI capabilities and supplier repositioning as potential sources of buyer confusion. NIST’s Generative AI Profile identifies risks including fabricated information, privacy leakage and problematic human reliance on AI output. ([verdantix.com](https://www.verdantix.com/insights/blog/riding-the-wave-of-capability-and-strategy-changes-in-the-grc-industry))

Use a buyer-held-out evaluation set and measure:

- Whether cited evidence actually supports each material conclusion.
- Whether inaccessible or deleted evidence can influence an answer.
- Handling of contradictory, obsolete and missing information.
- Human review and correction time, including rejected outputs.
- Ability to prevent or reverse unauthorised workflow changes.
- Data retention, training use, subprocessors and disablement.

Set acceptance thresholds before testing. A clean sample is not proof that future outputs will always be accurate. Retain input versions, source versions, relevant configuration and outputs; do not demand identical wording as the sole definition of reproducibility.

For high-consequence decisions, retain an accountable human approval step. This is the desk’s proposed control response to the confabulation and human–AI interaction risks described by NIST. ([nvlpubs.nist.gov](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf))