SWANK AI Guidance Note 14
Incident Governance · Error Correction · Operational Resilience

Core Standard

CORRECT THE RECORD. UNDERSTAND THE SYSTEM.

Effective AI governance requires more than trying to prevent failure.

It also requires the ability to:

identify

contain

correct

understand

and

learn from

AI-related incidents when they occur.

Purpose

AI governance often concentrates on preventing errors.

Prevention matters.

But errors are inevitable in complex systems.

Operational resilience therefore depends upon what happens after something goes wrong.

A well-governed organisation should be able to determine:

what happened;
who or what was affected;
whether the error remains active;
whether it entered other records or systems;
whether a current decision relies upon it;
how it can be contained;
how the record can be corrected;
and what should change to reduce recurrence.

An AI incident should not end with the statement:

“The AI was wrong.”

That describes the event.

It does not explain the system.

What May Constitute an AI Incident?

An AI-related incident may include:

fabricated factual information;
materially inaccurate summarisation;
incorrect classification;
biased output;
unintended automated action;
inappropriate disclosure of confidential information;
loss of source traceability;
failure to preserve disputed information;
inappropriate reliance on AI without meaningful human review;
incorrect recommendations affecting individuals;
propagation of an earlier AI-generated error;
failure to update an outdated conclusion;
or use of AI outside an approved operational scope.

Not every mistake requires the same response.

Incident handling should remain proportionate to:

consequence;
recurrence risk;
affected population;
reversibility;
evidence quality;
and the role the AI played.

Incident Is Broader Than Model Failure

An AI incident does not necessarily mean that the model itself malfunctioned.

The failure may instead arise from:

incomplete input data;
ambiguous instructions;
poor prompt design;
inappropriate use case;
inadequate human verification;
weak source traceability;
workflow design;
automation bias;
poor access control;
incorrect configuration;
outdated information;
or failure to propagate a correction.

The technical system may have behaved exactly as designed while the surrounding institutional process failed.

This distinction matters because different causes require different remedies.

Immediate Response

Where a material AI-related incident is identified, organisations may need to answer several immediate questions.

1. What Happened?

Identify the specific error or event.

Avoid replacing a precise description with a generic statement that “AI failed.”

2. Who or What Was Affected?

Determine whether the incident affected:

an individual;
multiple individuals;
a decision;
a record;
a workflow;
an internal process;
a public-facing service;
or another AI system.

3. Is the Error Still Active?

Determine whether the system continues to:

generate the same error;
rely on the same inaccurate information;
reproduce the classification;
expose the same data;
or influence current decisions.

4. Has the Error Spread?

Check whether it has entered:

case records;
summaries;
reports;
dashboards;
decision records;
downstream AI systems;
correspondence;
or other operational repositories.

5. Does Any Current Decision Rely Upon It?

An incorrect output may already have influenced consequential action.

That downstream effect may require separate review.

Containment

Containment should focus on stopping further operational consequences while the incident is understood.

Possible actions may include:

suspending a particular AI use;
pausing an affected workflow;
withdrawing an incorrect output;
restricting access;
flagging affected records;
preventing further automated propagation;
escalating for human review;
or temporarily reverting to a human fallback process.

The appropriate response depends upon the consequence and type of incident.

Not every error justifies shutting down an entire AI system.

But a material active error should not continue simply because investigation is incomplete.

Correction Should Propagate

Correcting the original AI output may not be sufficient.

If the inaccurate information has already entered:

case records;
reports;
summaries;
dashboards;
recommendations;
decisions;
later prompts;
retrieval systems;
or other AI-assisted workflows,

the error may remain operationally active.

Organisations should therefore consider:

Where did this information travel?

and

What now depends upon it?

Material corrections should be considered across the relevant information chain.

Correction Is Not Necessarily Deletion

Different incidents may require different forms of correction.

An inaccurate record may need:

amendment;
annotation;
supplementation;
withdrawal;
replacement;
or preservation alongside a visible correction.

Deleting an original record may sometimes remove important audit history.

A good correction system should preserve enough context to show:

what was originally recorded;
what was found to be wrong;
what the corrected position is;
and when the correction occurred.

Root-Cause Analysis

Incident analysis should move beyond the immediate output.

The useful question is not only:

What went wrong?

It is:

Why was the institution able to produce, rely upon or propagate the error?

Possible causes include:

Input Failure

The system received incomplete, incorrect or poorly structured information.

Model Limitation

The AI system was not capable of reliably performing the task.

Workflow Failure

The output entered a process without sufficient review.

Human Verification Failure

A person relied upon the output without checking relevant evidence.

Use-Case Failure

The system was used for a purpose beyond what it was approved or suitable for.

Traceability Failure

The underlying source could not be identified.

Governance Failure

Responsibility for review, escalation or correction was unclear.

Automation Bias

Staff gave excessive weight to the AI-generated output.

Summary-to-Source Drift

An earlier AI-generated interpretation was repeatedly copied until it appeared established.

The corrective action should match the actual cause.

Individual Error vs Recurring Failure

One incident does not necessarily mean that an entire AI system is unsuitable.

Likewise, repeated incidents should not automatically be dismissed as unrelated mistakes.

Governance should distinguish between:

Individual Error

A one-off mistake with limited consequence and low recurrence risk.

Recurring Failure Mode

A similar error appears repeatedly under comparable conditions.

Systemic Design Weakness

The workflow, governance or model use predictably creates the risk.

Inappropriate Use Case

The technology is being used for a task it cannot perform reliably enough for the consequence involved.

These categories require different responses.

Learning Without Overreaction

Good governance should avoid two opposite errors.

Underreaction

Treating repeated or consequential AI failures as isolated mistakes.

Overreaction

Abandoning useful AI because one correctable incident occurred.

The appropriate response should consider:

severity;
recurrence;
reversibility;
cause;
available safeguards;
and whether the system can be improved.

The objective is disciplined learning rather than reflexive confidence or reflexive rejection.

Incident Reporting

Organisations may benefit from a defined AI incident-reporting route.

A useful report may identify:

date;
system or model involved;
use case;
incident description;
affected records or people;
source of detection;
immediate containment;
evidence reviewed;
known downstream effects;
corrective action;
responsible owner;
review status;
and lessons identified.

Incident reporting should be proportionate.

The objective is to create enough visibility to support governance and learning.

Who Can Report an Incident?

Incident routes should not depend only upon technical teams.

Relevant concerns may be identified by:

end users;
professional staff;
administrators;
managers;
affected individuals;
auditors;
compliance teams;
vendors;
or people reviewing downstream records.

A person who encounters an apparent AI error should know:

where to report it

and

who has authority to act.

Incident Ownership

Every material incident should have identifiable ownership.

Relevant questions include:

Who receives the report?
Who determines severity?
Who can pause the system?
Who investigates?
Who corrects records?
Who reviews downstream impact?
Who communicates with affected people?
Who decides when the system can resume?
Who records lessons learned?

Responsibility should not disappear across multiple teams.

Human Review After an Incident

Where an AI incident affects a consequential decision, human review may need to extend beyond the technical error.

For example, reviewers may need to ask:

Did the incorrect output affect a decision?
Would the outcome have been different without it?
Was other evidence available?
Should the decision be reconsidered?
Does the affected person need to be informed?
Are downstream records still relying upon the same information?

Correcting the technology alone may not correct the institutional consequence.

Incident Severity

Organisations may classify AI incidents according to consequence.

Relevant factors may include:

number of people affected;
severity of harm;
sensitivity of information;
whether the error influenced a consequential decision;
recurrence risk;
reversibility;
whether the incident remains active;
and whether legal, contractual or regulatory obligations may be engaged.

A harmless drafting error and an incorrect safeguarding classification should not receive the same governance response.

Incident Trends

Individual incident records can become valuable governance data.

Organisations may examine:

recurring model errors;
common verification failures;
frequent override patterns;
repeated data-quality problems;
recurring workflow failures;
particular user groups affected;
repeated incidents after model changes;
and trends in correction requests.

The purpose is to identify whether apparently separate incidents share a common cause.

Error Correction as Governance Data

Corrections are not merely administrative housekeeping.

Repeated corrections may indicate:

poor source quality;
weak summarisation;
ambiguous workflow design;
inadequate human review;
unreliable model behaviour;
ineffective training;
or weak governance controls.

An institution that tracks correction patterns may identify problems before they become major incidents.

Model Change and Incident Review

External AI systems change over time.

Following a significant incident, organisations may need to determine whether:

the underlying model changed;
configuration changed;
prompts changed;
data sources changed;
staff use changed;
or the use case expanded.

A system that previously performed acceptably may behave differently after operational or vendor change.

Incident analysis should therefore consider system history.

Communication With Affected People

Where an AI-related error materially affects a person, appropriate governance may require clear communication.

Depending upon the context, this may include:

acknowledging the error;
identifying what information was inaccurate;
explaining whether a decision was affected;
providing the corrected position;
explaining what review will occur;
and identifying how further concerns can be raised.

Technical correction should not leave affected people unable to understand what happened.

Testing After Correction

A corrective action should be tested.

Relevant questions include:

Does the system still produce the error?
Has the correction reached downstream records?
Do users understand the new process?
Can the same failure still occur under slightly different conditions?
Has the model or workflow been retested under contradiction and uncertainty?
Did the corrective action create a new operational problem?

Closing an incident should require more than documenting that action was taken.

The organisation should understand whether the action worked.

Incident Closure

An incident may be ready for closure when, proportionately:

the active error is contained;
affected records are corrected;
material downstream effects are reviewed;
root cause is sufficiently understood;
corrective action is implemented;
recurrence risk is reassessed;
relevant stakeholders are informed;
and lessons are incorporated into governance.

Closure should reflect actual resolution rather than administrative completion alone.

Questions for Organisations

When an AI-related incident occurs, organisations may ask:

What exactly happened?
Who or what was affected?
Is the error still active?
Where has the error propagated?
Does any current decision rely upon it?
Can the error be contained?
Who owns the incident?
What was the root cause?
Is this an isolated mistake or recurring failure mode?
Have downstream records been corrected?
Should any consequential decision be reconsidered?
What governance change is required to reduce recurrence?

SIAAF Relevance

This Guidance Note principally relates to:

Domain 03 — Evidence & Traceability

Can the organisation identify where the incorrect information originated and where it travelled?

Domain 05 — Escalation, Challenge & Contestability

Can errors be identified, raised, corrected and reconsidered?

Domain 06 — Risk, Harm & Operational Resilience

Can the institution contain, recover from and learn from AI-related failure?

It may also engage:

Domain 01 — Governance & Accountability

Where responsibility for incident ownership and corrective action must remain identifiable.

Domain 02 — Decision Integrity & Human Oversight

Where an AI error influenced a consequential human decision.

Domain 04 — Communication & Feedback Integrity

Where corrections and lessons must move through the organisation effectively.

SWANK AI Standard

CORRECT THE RECORD. UNDERSTAND THE SYSTEM.

Effective AI incident governance requires:

identify the error

contain the consequence

trace the information

correct affected records

reconsider affected decisions

find the cause

test the correction

and

learn from recurrence.

The objective is not simply to establish that:

“the AI was wrong.”

The objective is to understand:

how the institutional system allowed the error to matter.

Related SWANK AI Guidance

Guidance Note 02 — AI Summarisation and Record Integrity

Guidance Note 08 — Reassessment Alongside Escalation

Guidance Note 10 — Governing AI Vendors, Data and External Models

Guidance Note 13 — Automation Bias in Professional Decision-Making

Guidance Note 15 — Testing AI Under Contradiction and Uncertainty

Guidance Note 19 — Correction Rights in AI-Assisted Systems

SWANK AI

Independent AI & Institutional Assurance

We do not just review AI. We review the institutional systems responsible for governing it.