SWANK AI Guidance Note 10
Vendor Governance · Data Governance · Operational Risk

Core Standard

GOVERN THE SYSTEM, NOT THE LABEL

“AI” is too broad a category for meaningful approval.

Governance should attach to a specific:

system

purpose

data environment

human workflow

and

level of consequence.

Purpose

Organisations often assess what an AI system can do before asking what happens to the information placed inside it.

Vendor governance should address both.

An AI system is not only a model.

It also exists within a wider:

technical environment;
contractual environment;
data environment;
security environment;
organisational workflow;
and governance structure.

An organisation therefore needs to understand not simply the capabilities of an AI product, but the conditions under which it will be used.

Approval Should Be Use-Case Specific

An organisation should avoid approving “AI” as a single general category.

A system that is appropriate for:

brainstorming;
drafting public-facing text;
internal research support;
or low-consequence productivity

may not be appropriate for:

confidential information;
sensitive records;
regulated data;
safeguarding material;
healthcare information;
legal material;
consequential decision support;
or high-stakes administrative processes.

Approval should attach to a defined product and defined use case.

Questions Before Adoption

Before approving an external AI system, organisations should understand:

what information the system receives;
where that information is processed;
where it is stored;
how long it is retained;
whether information may be used for model training;
whether subcontractors process data;
what security controls apply;
how access is controlled;
how data can be deleted;
what audit records exist;
and what happens following an incident.

The fact that a system is commercially available does not establish that it is suitable for every institutional use.

Understand the Data Flow

AI governance should include a clear understanding of how information moves through the system.

Relevant questions include:

What data enters the system?

Where does it go?

Who can access it?

How long does it remain there?

Does the provider use it for any secondary purpose?

Can the organisation retrieve or delete it?

Does another provider or subcontractor receive it?

Without that information, an organisation may understand the user interface while remaining unclear about the actual data environment.

Data Classification

Organisations may benefit from distinguishing between different categories of information.

PUBLIC

Information already appropriate for public disclosure.

INTERNAL

Ordinary organisational information not intended for unrestricted publication.

PERSONAL

Information relating to identifiable individuals.

SENSITIVE

Information that may include:

special-category data;
safeguarding information;
medical information;
financial information;
confidential records;
privileged information;
restricted material;
or other information requiring heightened protection.

Different categories may require different:

systems;
permissions;
safeguards;
approval processes;
or prohibitions.

A tool approved for public information may not be suitable for sensitive institutional records.

Data Minimisation

Organisations should consider whether the AI system requires all of the information users may be tempted to provide.

Relevant questions include:

Does the system need identifiable information?
Can material be anonymised or minimised?
Is the entire document necessary?
Can sensitive information be removed before processing?
Is the data proportionate to the task?
Are staff trained to recognise what should not be entered?

Convenience should not automatically determine the amount of information shared with an external system.

Model Limitations

Vendor assessment should also examine known system limitations.

These may include:

hallucination risk;
accuracy limitations;
bias;
prompt sensitivity;
inability to access current information;
unavailable source material;
difficulty explaining particular outputs;
inconsistent behaviour;
dependence on external services;
or limitations specific to the intended use case.

A capable model may still be inappropriate for a particular institutional purpose.

The question is not:

Is this a good AI system?

It is:

Is this system suitable for this task, using this data, within this workflow, at this level of consequence?

Model and Version Changes

External AI systems can change.

A vendor may update:

the underlying model;
safety controls;
context limits;
retrieval systems;
system prompts;
data practices;
pricing;
integrations;
or other operational features.

An organisation may therefore approve a system based on one version or behaviour and continue using it after substantial changes occur.

AI governance should include periodic reassessment.

Operational Drift

A system may also change without the vendor changing the product.

Staff may begin using it for purposes outside the original approval.

For example:

Approved use: drafting ordinary internal communications.

Later use:

analysing confidential records;
summarising sensitive case material;
making recommendations;
assessing individuals;
or supporting consequential decisions.

The technology may be unchanged.

The operational risk has changed.

Governance should therefore monitor use-case drift as well as model change.

Questions for Periodic Reassessment

Organisations may ask:

Is this still the same operational system we originally approved?
Has the underlying model changed?
Have the capabilities changed?
Have known risks changed?
Has the vendor changed its data practices?
Are staff using the tool for additional purposes?
Has sensitive information entered the workflow?
Have incidents or recurring errors emerged?
Does the original approval remain proportionate?

Approval should not become permanent simply because the initial assessment occurred.

Contractual Accountability

Vendor arrangements should clarify responsibility for matters such as:

security;
confidentiality;
data loss;
incident notification;
service changes;
model changes;
service availability;
audit rights;
deletion obligations;
subcontractor use;
termination;
and transition arrangements.

Organisations should understand where vendor responsibility ends and institutional responsibility begins.

A contract may allocate responsibility between parties.

It does not eliminate the organisation’s own governance obligations.

Vendor Responsibility Does Not Replace Institutional Responsibility

An organisation should not assume that:

“the vendor approved it”

“the provider is responsible for the AI”

removes institutional accountability.

The organisation still determines:

why the system is being used;
what data is entered;
which staff can use it;
what outputs are relied upon;
where human review is required;
and whether the use remains appropriate.

The vendor provides a system.

The institution governs its use.

Human Workflow

AI governance should examine what happens around the model.

Relevant questions include:

Who enters information?
Who reviews the output?
What verification is required?
Can a user override the result?
Does the system trigger further action?
What decisions remain human?
Where is accountability recorded?
What happens when the system is wrong?

Technical capability should not be considered separately from the human workflow in which it operates.

Source Traceability

Where an external AI system retrieves, summarises or generates information, organisations should understand whether users can inspect the underlying sources.

Questions may include:

Does the system preserve source references?
Are citations retrieved or generated?
Can users inspect the original material?
Can claims be verified?
Does the system distinguish sourced information from generated interpretation?
Can corrections be linked back to affected outputs?

Source visibility becomes increasingly important as the consequence of the output increases.

Confidentiality

External AI systems may create particular concerns where users enter:

personal information;
commercial information;
medical information;
safeguarding records;
legally privileged material;
unpublished research;
or other confidential material.

Governance should define clearly:

which systems are approved;
what information is permitted;
what information is prohibited;
who can access the systems;
and what safeguards apply.

Staff should not be required to make these judgments from guesswork.

Security and Access

AI systems should also be considered within the organisation’s wider access-control environment.

Questions may include:

Who can create accounts?
Are organisational accounts required?
Is multi-factor authentication available?
Can access be revoked?
Are user activities logged?
Can administrators review use?
Are integrations appropriately controlled?
Does the system connect to internal documents or databases?
What happens when a staff member leaves?

AI adoption should not create an ungoverned route into institutional information.

Incident Management

Organisations should know what happens if:

confidential information is exposed;
an account is compromised;
the system generates harmful or materially inaccurate output;
service availability fails;
data cannot be retrieved;
a vendor changes terms unexpectedly;
or an external dependency becomes unavailable.

Relevant arrangements may include:

incident notification;
internal escalation;
containment;
suspension of use;
investigation;
correction;
and reassessment.

Vendor governance should connect to the organisation’s wider incident-response structure.

Exit Planning

Organisations should consider what happens if an external AI service:

is withdrawn;
becomes unsuitable;
changes price;
changes its data practices;
experiences prolonged outage;
changes ownership;
or creates unacceptable risk.

Operational resilience may require:

alternative workflows;
exportability of records;
preserved source information;
human fallback processes;
substitute systems;
and clear termination arrangements.

Adoption should therefore include thinking about how the organisation could stop using the system.

Avoid Dependency Without Visibility

An AI system may gradually become embedded in:

communications;
knowledge management;
analysis;
record preparation;
workflow;
staff productivity;
or decision-support processes.

As dependency increases, organisations should understand:

what would happen if access disappeared;
what records would remain;
what institutional knowledge depends upon the vendor;
and whether critical processes can continue.

Convenience can become operational dependency over time.

That dependency should be visible.

Procurement Before Deployment

AI procurement should involve more than feature comparison.

Organisations may need to examine:

intended purpose;
users;
information type;
consequence level;
model limitations;
data environment;
security;
contractual terms;
human oversight;
source traceability;
incident response;
reassessment;
and exit arrangements.

The procurement question should be:

Can this system be governed appropriately in the environment where we intend to use it?

Questions for Organisations

Before adopting or renewing an external AI system, organisations may ask:

What precise use case are we approving?
What data will enter the system?
Where is that data processed and stored?
Is any information used for model training?
Which subcontractors or third parties are involved?
What are the system’s known limitations?
Can important outputs be traced to source material?
Who reviews consequential outputs?
What happens when the model changes?
How will we detect unapproved use-case expansion?
What happens following an incident?
Can the organisation exit the service without losing critical information or operational capability?

SIAAF Relevance

This Guidance Note principally relates to:

Domain 01 — Governance & Accountability

Who owns the approval, use and oversight of the external AI system?

Domain 02 — Decision Integrity & Human Oversight

How are vendor outputs incorporated into human workflows and consequential decisions?

Domain 03 — Evidence & Traceability

Can outputs, sources and system involvement be reconstructed?

Domain 06 — Risk, Harm & Operational Resilience

Does the organisation understand vendor, data, security, model-change and dependency risks?

It may also engage:

Domain 07 — AI Literacy & Organisational Readiness

Where users need to understand approved systems, prohibited data and appropriate use cases.

SWANK AI Standard

GOVERN THE SYSTEM, NOT THE LABEL

Approval should attach to:

a specific system

a defined purpose

an identified data environment

a human workflow

a level of consequence

and

clear institutional responsibility.

Organisations should know:

what the system does

what information it receives

where that information goes

who remains responsible

what happens when the system changes

and

how the organisation can stop using it.

“AI” is too broad a category for meaningful governance.

Related SWANK AI Guidance

Guidance Note 03 — Human Oversight in AI-Assisted Decision Systems

Guidance Note 07 — Public-Sector AI and Accessibility

Guidance Note 14 — AI Incident Reporting and Error Correction

Guidance Note 17 — Source Traceability by Design

Guidance Note 20 — Governing AI in High-Stakes Environments

Guidance Note 23 — AI Procurement Before Deployment

SWANK AI

Independent AI & Institutional Assurance

We do not just review AI. We review the institutional systems responsible for governing it.