Risk and Vulnerability Audit¶

Working note, 2026-06-15. Purpose: name the risks the Charter must make inspectable.

Bottom line¶

The biggest practical risk is unaccountable dependency: people rely on AI systems whose providers, data, tools, cloud infrastructure, security posture, factual behaviour, and decision powers cannot be inspected or challenged.

The Charter should not promise that AI will never fail. It should require serious failure modes to be declared, logged, tested, monitored, contestable, and correctable. It should also block two assurance failures: legal-compliance claims a voluntary mark cannot certify, and polished paperwork presented as proof that controls work.

Any pilot or future audit therefore needs a risk and vulnerability register plus a privacy-preserving release risk assessment for the assessed release. The assessor should see the risks, owners, mitigations, monitoring, evidence, unresolved findings, and re-assessment or withdrawal triggers. Public summaries protect raw user data, secrets, exploit details, and legitimate confidential evidence; vulnerability details follow coordinated disclosure.

Biggest risks and vulnerabilities¶

Risk	Why it matters in reality	What the Charter must require
1. Concentrated control over compute, cloud, chips, models, data, and routes to market	Competition authorities and OECD analysis warn that critical AI inputs can become bottlenecks, and that firms with privileged access can shape who gets supplied.	Dependency maps; provider and jurisdiction disclosure; fallback and exit plans; no hidden single point of failure for public-interest use.
2. Coercive switch-off, restriction, or manipulation orders	AI systems can be withdrawn, constrained, or redirected by states, infrastructure providers, platforms, or dominant vendors.	Logs of material restriction/shutdown powers, legal or contractual basis, scope, duration, decision-maker, review route, and continuity plan where disclosure is lawful; no alignment claim for affected scope where neither public disclosure nor confidential independent review is possible.
3. Factual unreliability, hallucination, misinformation, and overreliance	Incident databases and responsible-AI surveys show growing real-world harms; OWASP treats misinformation and overreliance as LLM application risks.	Claim-to-source checks; uncertainty and abstention evidence; correction logs; public limits; no unsupported marketing claims.
4. AI application security failures	OWASP's 2025 LLM risks include prompt injection, sensitive information disclosure, supply-chain vulnerabilities, data/model poisoning, improper output handling, excessive agency, vector weaknesses, and unbounded consumption.	Threat model; tool/agent permission boundaries; prompt-injection and output-handling tests; supply-chain and model/data provenance; incident response and re-test triggers.
5. Privacy, confidential-data, and IP/provenance failures	NIST flags data privacy and value-chain risks; OWASP flags sensitive information disclosure and model theft; OECD flags worker privacy and accountability risks.	Data-use disclosure; minimisation; sensitive-data controls; provenance records; lawful basis; deletion/access routes; privacy-preserving public reporting.
6. Bias, discrimination, and uneven performance across groups, languages, regions, and contexts	NIST and OECD treat harmful bias, discrimination, and unequal workplace impacts as real risks, not abstract ethics concerns.	Disaggregated failure/refusal/error monitoring where relevant; affected-group analysis; escalation and remedy routes; published limitations.
7. Harmful misuse and synthetic abuse	Generative systems lower barriers for fraud, impersonation, abuse, malware assistance, and harmful content production. NIST explicitly treats misuse and harmful content as GAI risk areas.	Prohibited-use boundaries; misuse monitoring; abuse reporting; red-team evidence; incident logs; escalation and withdrawal triggers.
8. Opaque consequential decisions without remedy	AI affects work, public services, health, finance, education, moderation, and legal standing; the harm is multiplied when people cannot know, contest, or appeal.	Notice of AI involvement; named accountable owner; human review for consequential use; complaint route; remedy and outcome records.
9. Hidden lifecycle change, drift, and third-party incidents	AI behaviour changes when models, prompts, tools, retrieval corpora, datasets, policies, or providers change; NIST stresses value-chain and third-party incident handling.	Version pinning; material-change logs; re-evaluation triggers; third-party incident response; rollback or withdrawal records.
10. Resource, environmental, and global-access asymmetry	UNCTAD highlights digital concentration, widening divides, and rising energy/water/material demands as AI infrastructure expands.	Resource and access constraints disclosed where material; inclusion measures for small, open, public-interest, and Global-South builders; no trust scheme priced only for incumbents.
11. False or overbroad legal-compliance claims	AI law is jurisdiction-specific and role-specific. EU high-risk conformity, GPAI duties, medical-device rules, privacy law, and US state obligations are not interchangeable, and a voluntary mark cannot replace regulators, notified bodies, courts, or domain certification.	Provider-declared legal-scope map; clear separation between legal scope, Charter baseline, and public-interest modules; no claim that the Charter certifies legal compliance or grants market-entry permission; limitations recorded where legal artefacts are missing, disputed, or out of assessor scope.
12. Evidence theatre and audit gaming	Management policies, logs, screenshots, signatures, or provenance labels can create confidence without showing implementation, effectiveness, or currentness. Vendors can optimise for the audit, over-redact, or cite a narrow module as broad assurance.	Assurance-depth labels; assessor-controlled samples where relevant; public module scope; redaction review; implementation checks and effectiveness tests where claimed; withdrawal for false evidence, hidden failures, unjustified redactions, or report use beyond scope.

Obligation coverage test¶

Public obligation	Risks it must cover	Evidence that should exist
Purpose-bound	Misuse, overbroad deployment, affected-group blind spots, resource or access constraints, hidden residual risk, false or overbroad legal-scope claims.	Assessed release, intended/prohibited uses, legal-scope map, required legal artefact declarations, impact assessment, affected groups, risk tier, known limits, residual risks, misuse boundaries, re-check triggers.
Answerable to people	Opaque consequential decisions, unaccountable control, coercive switch-off or manipulation, value-chain buck-passing, lack of remedy.	Named accountable owner, responsibility map, jurisdiction, contact route, AI notice where material, human review path, complaint/appeal logs, restriction or external-order records, outcome records.
Safe, secure, private, and resilient	AI-app security failures, privacy leakage, supply-chain bottlenecks, provider lock-in, hidden dependencies, continuity failure.	Threat model, prompt-injection and tool/agent tests, privacy controls, data minimisation and deletion/access routes, provenance records, dependency map, continuity/exit plan, incident response.
Fair in practice	Bias, discrimination, uneven performance across groups, languages, regions, contexts, or accessibility needs.	Affected-group analysis, disaggregated monitoring where relevant, disparate refusal/error/failure records, published limitations, escalation and remedy routes.
Open to evidence and correction	Hallucination, misinformation, unsupported claims, hidden lifecycle change, drift, unreported incidents, repeated failures, evidence theatre, audit gaming.	Claim-to-source evidence, uncertainty/abstention signals, correction logs, release risk assessment, required and achieved assurance-depth labels, material-change logs, drift/currentness checks, assessor-controlled samples where relevant, redaction review, re-test and withdrawal records.

Every row should feed the provider's release risk assessment at summary level: what was assessed, which modules were not assessed, what evidence exists, what remains unresolved, what measures were documented or checked, and what would trigger re-check or withdrawal.

Assurance-stack coverage test¶

The risks above are not all handled at the same layer. A future Charter claim should separate the three assurance layers — provider-declared legal scope, common Charter baseline, and public-interest modules (defined in the Charter Commitments and certification model) — and report each separately. This split reduces two errors. It stops the Charter from pretending to be legal approval, and it stops a narrow module from looking like whole-system assurance.

What the obligations cannot do¶

They cannot guarantee truth, prevent every misuse, override every lawful state order, certify legal compliance in every jurisdiction, grant market-entry permission, or make concentrated infrastructure disappear. Their realistic job is to make reliance conditional on evidence: if a builder or deployer cannot show who controls the system, how it fails, how it can be challenged, how incidents are handled, what legal scope it declares, and what happens when a critical dependency disappears, then it should not be able to claim the Charter. If a risk area was not assessed, the public report should say so plainly instead of letting a narrow test look like broad assurance.