The AI Vendor Risk Questionnaire You Should Actually Be Sending

The AI Vendor Risk Questionnaire You Should Actually Be Sending

Introduction

Most vendor questionnaires today feel like relics of a pre-AI world: checkbox security supplements and boilerplate privacy answers that give procurement teams the illusion of safety. That's exactly what happened at a mid-sized healthcare payer we work with. Their procurement team was rubber-stamping AI vendors because the "AI security supplement" they relied on was a 12-question addendum that hadn't been updated since 2019.

Then a vendor quietly trained a model on customer data that included protected health information (PHI), in a way that violated the payer's business associate agreement (BAA). The payer avoided a public regulatory enforcement action by luck and quick containment - but the incident destroyed trust, triggered internal investigations, and pushed leadership to rebuild their AI vendor due diligence from the ground up.

They redesigned the questionnaire around six practical themes that map to real operational risks-training data lineage, model behavior controls, evaluation evidence, sub-processor disclosures, customer data isolation, and incident notification. Within a single procurement cycle, two vendors were dropped on the first pass and three tightened contractual terms to keep the business.

This article explains why most AI questionnaires fail, what a real one looks like, sample questions to use, red flags that should kill a deal, and how to bolt this into your existing third-party risk management (TPRM) workflows. The goal: govern AI in a way that protects patients, customers, and your balance sheet - without kneecapping innovation.

Why most AI questionnaires are useless

- They ask the wrong questions. Security-centric questionnaires focus on encryption, firewalls, and SOC attestations - all important - but ignore how AI models are trained, updated, and monitored.
- They rely on vendor-supplied buzzwords. "We don't train on customer data" is easy to say and often undefined.
- They're static. AI is evolving fast; controls that made sense in 2019 don't address model retraining, third-party models, or prompt-based data leakage.
- They treat AL/ML like traditional software. Models have behaviors that emerge from training data and design choices; governance must assess behavior, not just components.

A better model: six themes that map to tangible risk controls

We rebuilt the payer's AI questionnaire around six themes aligned to frameworks like the NIST AI RMF (Govern, Map, Measure, Manage) and emerging regulatory expectations (EU AI Act's documentation requirements, ISO/IEC 42001's management system approach). Below are plain-English explanations, sample questions, and red flags to help you operationalize each theme.

1) Training data lineage - "Where did this model learn what it knows?"

Why it matters: Risk to privacy, intellectual property, and compliance often starts with data. Lineage shows sources, consent status, preprocessing, and retention - essential in regulated sectors like healthcare (HIPAA/BAA considerations), finance, and telecom.

Sample questions:
- List all data sources used to train or fine-tune the model (vendor internal, public, licensed, customer data). For each source, provide provenance, date ranges, and licensing/consent status.
- For any customer data used, describe the contractual basis (e.g., BAA, DPA) and show documented customer consent or permission to use that data for training.
- Describe preprocessing steps applied to raw data (de-identification, tokenization, augmentation). Provide tools and scripts or hashes where possible.
- What records do you maintain to support a forensic audit of training data lineage? How long are these records retained?

Red flags that should kill a deal:
- Vendor refuses to disclose training data sources or provides only generic answers.
- Vendor admits to using customer data for model improvement without documented contractual consent.
- No records or poor retention that prevent auditing lineage.

2) Model behavior controls - "How do you keep the model behaving the way we expect?"

Why it matters: Models can hallucinate, be biased, or produce harmful outputs. Controls for prompting, guardrails, refusal mechanisms, and safety layers are the operational equivalent of access control for models.

Sample questions:
- Describe safety/guardrail mechanisms (e.g., refusal policies, content filters, toxicology checks). Include examples of prompts the model would refuse and how refusals are implemented.
- How do you tune the model for false positives vs. false negatives? Provide thresholds used in production and rationale.
- Explain the change management process for model updates, including rollback mechanisms and approval gates.
- For models delivering clinical or financial recommendations: describe any human-in-the-loop workflows and decision-making authorities.

Red flags:
- No documented safety strategy or ad hoc mitigation only.
- Inability to demonstrate rollback or staged rollout procedures.
- Refusal to allow human oversight where outputs materially impact customers.

3) Evaluation evidence - "Can you prove the model works and is safe for our use?"

Why it matters: Vendors must provide evidence that models were tested for accuracy, fairness, robustness, and privacy risks. Lab claims aren't enough - look for test artifacts, benchmark data, and continuous monitoring evidence.

Sample questions:
- Provide test suites and evaluation datasets used for pre-deployment validation. For each, include distributional characteristics and whether they reflect our population.
- Share performance metrics (precision, recall, AUC, calibration) and any subgroup analyses (race, age, clinical cohorts).
- Describe adversarial testing, red-teaming results, and mitigations for prompt injection or data extraction attacks.
- Provide monitoring dashboards or examples of production drift detection and alerting.

Red flags:
- No access to evaluation artifacts or only high-level marketing metrics.
- No subgroup or bias analysis for sensitive populations.
- No production monitoring or drift detection.

4) Sub-processor disclosures - "Who else touches our data or the model?"

Why it matters: Third parties may be co-training models, providing embeddings, or offering inference APIs. Sub-processors multiply risk and complicate compliance and auditability.

Sample questions:
- Provide a current list of sub-processors (cloud providers, model providers, data suppliers) and their roles. Include contractual flow-downs for security and compliance.
- For any third-party model components (LLMs, embeddings), provide versioning and change notification commitments.
- Do you permit sub-processors to further subcontract? If so, what controls apply and how are customers informed?

Red flags:
- Vendor refuses to identify sub-processors or claims "we may use third parties" without specifics.
- Critical sub-processors located in jurisdictions that conflict with data residency or sector rules.
- No contractual flow-downs or inadequate controls on subcontractors.

5) Customer data isolation - "Can you guarantee our data won't leak into other customers' training sets or outputs?"

Why it matters: Data isolation prevents data leakage during training, fine-tuning, inference logging, and telemetry. It's a must for PHI, financial data, and other sensitive information.

Sample questions:
- Describe data segregation architecture for storage, training, and inference. Are production logs stored separately per customer?
- Do you use customer data to improve shared models? If yes, describe opt-in/opt-out mechanisms, and how customer data is isolated during training.
- How are prompts and inference records retained? Can customer data be purged on request?

Red flags:
- Default practice of aggregating customer data for model improvement with no opt-out.
- No per-customer data tagging, retention policy, or purge capability.
- Evidence of prior cross-customer leakage incidents.

6) Incident notification and forensics - "Will we know if something goes wrong, and can we investigate?"

Why it matters: Timely detection and transparent notification are critical for containment, regulatory reporting, and maintaining trust. AI incidents often require model-specific forensics (training snapshots, inference logs).

Sample questions:
- Commit to notification timelines for security breaches, privacy incidents, and model behavior incidents (e.g., within 48/72 hours). Describe the channels and contacts.
- What telemetry do you retain (inference logs, prompts, model versions) to support a forensic investigation? Retention period?
- Describe a recent incident, your response, and post-mortem learnings (redacted as appropriate).

Red flags:
- No incident notification SLA or refusal to commit to a notification timeline.
- Insufficient telemetry to investigate model behavior or training changes.
- History of undisclosed incidents or poor remediation.

Red flags that should kill a deal (summary)

Any of the following should trigger a stop to negotiations unless remediated:
- Proof vendor used customer PHI for training without consent or BAA alignment.
- Vendor refuses to provide training data lineage or evaluation evidence.
- No contractual controls for sub-processors or data residency violations.
- Inability to isolate customer data or to purge it on demand.
- No incident notification commitment or inability to support forensics.
- No human-in-the-loop or control mechanisms where outputs materially affect outcomes.

How to integrate this into existing TPRM workflows

1. Replace single-stage questionnaires with a tiered approach
- Stage 1 (RFI): Quick red-flag checks mapped to the six themes. Use to screen vendors before deeper diligence.
- Stage 2 (DD): Full AI questionnaire for high-risk vendors or those shortlisted.
- Stage 3 (Onboarding & Contracting): Contract negotiations, SLAs, and attestations based on questionnaire gaps.

2. Map to risk tiers
- Define risk tiers (low/medium/high) based on use case impact (e.g., administrative chatbot vs. clinical decision support).
- Require full AI due diligence for medium/high risk; light touch for low risk.

3. Score and playbook
- Create a scoring rubric where each theme receives a score and thresholds map to remediation items, contract clauses, or rejection.
- Example playbook items: restrict data used for training, require contractual right to audit, add BAA clauses, or require model explainability artifacts.

4. Make it cross-functional
- TPRM owns the process, but involve legal (contracting and privacy), security, product risk, clinical safety (healthcare), and the business owner. For high-risk models, include an internal AI governance committee.

5. Contractual meat
- Translate controls into contractual terms: explicit prohibition on using customer data for training without consent, right to audit model lineage, defined incident notification SLAs, sub-processor disclosure/flow-downs, and termination for violation.

6. Continuous monitoring
- Treat AI vendors like dynamic risks. Require regular evidence (quarterly attestation, drift reports) and trigger re-assessment on major model changes or when the vendor announces new training data sources or model versions.

Governance as an enabler, not a blocker

Good AI governance reduces friction by making expectations clear up front. When the payer rewrote their questionnaire, procurement stopped getting surprised by last-minute model changes. Two vendors dropped because they couldn't meet basic isolation and lineage requirements - a difficult but necessary outcome. The others tightened practices and contractual commitments, which made the payer confident to continue adoption. Governance enabled the organization to move faster on safe terms.

This approach is aligned with practical frameworks:
- NIST AI RMF: Use the Govern/Map/Measure/Manage lifecycle to structure evidence and controls.
- EU AI Act: For high-risk systems, document technical documentation and conformity assessments early.
- ISO/IEC 42001 (AI management system): Build repeatable processes that can be internalized and audited.

Conclusion - the takeaway for boards and executives

If your third-party AI questionnaire hasn't been updated since 2019, you're not doing vendor due diligence - you're doing wishful thinking. Replace checklist theater with a focused, risk-based questionnaire around the six themes above: training data lineage, model behavior controls, evaluation evidence, sub-processor disclosures, customer data isolation, and incident notification.

Practical steps to start this week:
- Run a gap assessment of your current AI/ML vendors against the six themes.
- Introduce a two-stage AI vendor RFI for new deals and a remediation plan for existing vendors.
- Add contractual clauses for data use, audits, and incident SLAs.
- Stand up a cross-functional AI governance review for any medium/high-risk AI use.

Governance isn't about saying no. It's about getting the right information early so you can say "yes" confidently - and keep saying it as AI evolves.