AI vendor due diligence checklist: seven questions and the documents that answer them

AI vendor due diligence for assessment: seven questions to ask, and the documents that answer each — DPA, DPIA template, marking methodology, module notes.

An AI vendor due diligence checklist for an assessment platform should end in documents, not adjectives: who decides at each step, what each AI module does with which data, how AI-assisted marking was validated, what you can request about the models, and which contract governs the processing. NUADU answers each line in writing, with a documentation pack, a signable DPA and an AI Transparency Notice. The checklist comes first below, then the documents that answer it, then the people who read each part.

What should an AI vendor due diligence checklist include?

Seven questions, each answered by a document rather than a sales call. For an assessment platform they are:

  • Can a human override every AI output, and is the override recorded? The audit trail and the override workflow answer it.
  • What does each AI module do and not do, with which data, and who can contest it? The per-module notes answer it.
  • How was AI-assisted marking validated, including for bias? The marking methodology answers it.
  • What can we request about the models? The AI Transparency Notice answers it.
  • Which sub-processors touch the data, and what contract covers them? The sub-processor list and the DPA, both covered on the Trust & compliance page, answer it.
  • Is there support for our DPIA? The DPIA template, data-flow diagrams and retention policy answer it.
  • Where does the vendor stand on the EU AI Act? The notice answers it, and an EU AI Act briefing for assessment is on request.

On NUADU most of them arrive together as the documentation pack, available on request to QA, DPO, procurement and academic-board reviewers; the model questions go through the notice's request route. The reason it exists is the audit trail: every AI output, override and approver is captured end to end, and the pack is what lets a QA office, a regulator or a board trace a decision to a named human.

The trail records the instance. The pack describes the method.

What questions should you ask AI vendors?

Start with the AI itself, and ask for the answers in writing. Four questions do most of the work.

Who decides? On NUADU, a named educator at every decision point. The marker can override any AI score, the candidate can contest any released grade through your appeals process, and the audit trail supports both. A vendor that cannot name the human step for each module has answered the question too.

What does each module do? For each AI module, the pack states what it does, what it does not do, what data it uses, who decides and who can contest. The AI capabilities page gives the short form, each capability with what the human decides and what the AI never does; the pack gives the long one. Works councils and employee representatives ask the same kind of question — what the AI does, who decides, how a result is contested — and they are right to.

How was it validated? The marking methodology covers rubric application, validity evidence for AI-assisted marking, inter-rater agreement data, bias review, the marker override workflow and the candidate appeals workflow. Your QA office can read it against the marking policy it already applies. The limits are written down too. The notice names four kinds: accuracy, since a model can mis-score or miss errors; bias, since models can reflect biases in their training data and may treat some dialects, non-native speakers or writing styles less fairly; language, since performance varies by language and domain, with English inputs the most reliable; and audio, since speaking and pronunciation scores depend on microphone quality and background noise. NUADU monitors for bias and adjusts prompts and rubrics, and the monitoring results are available on request.

What can we request about the models? NUADU pins model versions per deployment; changes are tested before release and recorded in the internal change log. Data subjects and institutional deployers may request the model versions in use, the categories of system prompts (not the proprietary text), bias-monitoring results, the DPIA summary and the current status of the formal EU AI Act obligations, and NUADU responds within one month. The AI Transparency Notice is where those rights are written down. What the Act itself asks of AI that evaluates learning outcomes is set out in EU AI Act high-risk rules for education.

What security questions should you ask AI vendors?

The ones a security reviewer asks of any hosted service, plus two that AI makes sharper: which companies besides the vendor touch the data, and on what terms. NUADU's answers are short and stated on its Trust page.

  • Controls: SOC 2-aligned security controls, which the Trust page describes as controls and not as a certification claim.
  • Data and access: encryption in transit and at rest, multi-factor authentication, role-based access control, and SSO via SAML or OAuth.
  • Recovery and availability: disaster recovery is built in, live support runs during every exam session, and contractual availability commitments with named responsibilities are agreed per deployment. NUADU does not publish a blanket uptime figure.
  • Sub-processors: a named list with each one's role, provided with the DPA and updated on change.
  • Contract: a DPA comes as a document to sign, not a questionnaire to fill in, and NUADU signs one with every customer.

Where the data is hosted, and whether the GDPR requires it to stay in the EU, is a question of its own; it is answered in whether GDPR requires data to be stored in the EU. Ask, too, how your data may be used beyond the service you are buying. That answer belongs in the DPA, and the DPA is where to read it. A technical one-pager is on the same on-request list as the pack.

Do you need a DPIA for AI tools?

Where the processing is likely to be high-risk, yes. Under Article 35 of the GDPR, a DPIA is expected before any processing that is likely to put people's rights and freedoms at high risk, particularly where new technologies are involved. Whether a given AI tool crosses that line is the controller's judgement, which in an institutional deployment means the institution's.

Is there a DPIA template for AI? NUADU's documentation pack includes one, with the data-flow diagrams, the retention policy and the sub-processor list for institutional review. NUADU supplies the template; the institution performs the DPIA, and the purpose, necessity and proportionality judgements stay with the institution. The UK ICO's guidance on DPIAs adds a useful reassurance: there is no definitive template you must follow, and the ICO offers a suggested one of its own. Deployers can also request NUADU's DPIA summary.

Some processing needs an assessment of its own. Recorded exam sessions are the usual case, and online proctoring and GDPR walks through that DPIA with the DPO in the room.

Who reads which part of the pack?

Each section has a first reader. Set the documents against the people who will sign off on them, rather than summarising the pack for everyone:

  • DPO: the DPIA template, data-flow diagrams, retention policy, sub-processor list and the DPA.
  • QA office: the marking methodology, validity evidence, inter-rater agreement data, bias review, and the override and appeals workflows.
  • Works council: the per-module notes, read module by module, with the named decision step behind each.
  • Procurement: the signable DPA, the sub-processor list and the security posture, with the labels read as written.
  • Academic board: the per-module notes and the methodology, read against your existing second-marking policy.
  • Regulator: the integrity-control documentation, the candidate due-process workflow and the audit trail, available on request.

For a tender, the sections map onto the questions as they are usually asked. For a DPIA, the template and the data-flow diagrams give the structure. Either way, the audit trail is what shows the documented method at work on a real session.

The pack is not meant to be read alone. The site offers a 30-minute review of every AI module, the human-accountable workflow, the audit trail and the documentation pack, and asks you to bring your QA officer, DPO or procurement reviewer. Request the pack first, so the questions are ready.

→ Talk to us about your assessment workflow (30 min)