How an AI Risk Review Works

An Independent AI Risk Review begins with a concrete decision that needs support. The review boundaries, required information and depth of analysis are then shaped around the relevant system, workflow and potential consequences.

The outcome is decision support within defined boundaries: a clearer account of what is supported, what remains uncertain and what should happen before the next consequential step.

Discuss Your AI System

Explore Services

The Review Starts With the Decision

The engagement is not determined by a generic checklist or a preselected technical test.

The initial conversation clarifies:

the next decision and its potential consequences;
the system’s current stage;
the workflow or business process affected;
the authority or autonomy involved;
the controls and unresolved questions that matter most;
the information currently available for review.

These factors shape the appropriate review route, boundaries and depth.

You do not need to select the final route before making contact. It is confirmed through professional judgment after the initial conversation.

Review Process

Define

1. Initial Conversation

The process begins with a short discussion of the system, its current stage and the next decision.

Detailed technical materials are not required at this point.

2. Decision Context and Review Boundaries

The relevant system boundary, workflows, responsible roles and decision conditions are defined.

The review scope may cover the wider system or focus on a specific workflow, component or control question. The agreed boundaries also make clear what will not be examined.

3. Focused Evidence Request

A focused request is prepared for the materials and access needed to examine the agreed questions.

This may include:

architecture and workflow documentation;
policies and operating procedures;
permissions, approval rules and configuration information;
test results and evaluation records;
sanitized logs or traces;
demonstrations, observations or stakeholder interviews.

The aim is not to collect every available document, but to establish a reliable basis for analysis.

Examine

4. System and Workflow Reconstruction

The available materials are used to build a practical view of how the relevant system and operating process are expected to work.

This may include:

components and data or context flows;
human and system decisions;
roles, ownership and escalation paths;
access, permissions and approval points;
monitoring and intervention mechanisms;
fallback and recovery arrangements.

The reconstruction also shows where the intended process still depends on assumptions or incomplete information.

5. Review and Controlled Testing

The review compares what is documented or reported with what can be observed or tested.

Where useful and agreed, a prototype, demonstration, controlled scenario or sanitized operational material may be used to examine a specific failure mode or control question.

Controlled testing strengthens a defined line of inquiry. It does not establish behavior across all production conditions.

6. Findings and Controllability

Relevant observations and documented conditions are translated into findings that connect:

the underlying technical or operational mechanism;
the potential effect on operations or business outcomes;
existing strengths and controls;
material gaps or unsupported assumptions;
actions that should take priority.

The review also examines whether incorrect actions can be prevented, unexpected behavior detected, the system interrupted when necessary and operations restored after failure.

Material limitations are recorded alongside the findings.

Deliver

7. Decision-Support Deliverable

The results are brought together in a concise written deliverable shaped around the agreed review boundaries.

It may include:

a practical reconstruction of the relevant system and workflow;
key findings, consequences and control limitations;
conditions relevant to the next operational step;
prioritized remediation or validation actions.

The output may take the form of an AI Risk Review Report, an Executive Decision Support Note or another clearly defined document appropriate to the engagement.

8. Prioritized Next Steps

Recommendations are ordered by their relevance to the decision ahead.

Where further confirmation is needed, the deliverable identifies what should be corrected, tested or observed.

A follow-up review is not automatically required. The next step depends on the findings, the actions taken and the remaining uncertainty.

Different Evidence Supports Different Conclusions

An AI Risk Review does not treat every document, statement or test result as equivalent.

Documented

What architecture documents, specifications, policies, procedures or system records state.

Documentation can clarify intended design, but does not by itself show that a control is implemented or consistently enforced.

Reported

What responsible stakeholders describe in interviews or written explanations.

It can clarify how the system is understood and operated, but may require confirmation from other sources.

Observed

What can be seen directly in a demonstration, workflow, accessible environment or recorded behavior.

Observation supports conclusions about the reviewed setting, but may represent only a limited scenario.

Synthetic or Controlled-Test

What is produced through a bounded scenario designed to examine a specific mechanism, failure mode or control.

It does not establish performance across every operating condition.

Representative Operational

Information produced under conditions sufficiently similar to the intended operating environment.

It can support conclusions about likely behavior when the similarities and remaining differences are understood.

Production Operational

Information from actual production use, such as relevant configurations, sanitized logs, traces, incidents or operating records.

Its value still depends on coverage, quality, traceability and the questions examined.

The review records which types were available, what each can support and where further confirmation remains necessary.

See the Evidence page for how reviewed material is also classified by review status.

Choosing the Right Route and Depth

AI System Design Risk Review

This route addresses intended architecture, workflows and planned controls.

It may rely on documentation, interviews and design materials, and may be strengthened through a prototype, demonstration or controlled scenario.

This is an evidence-enriched Design Review, not a separate third service.

Operational AI Risk Review

This route addresses actual or representative behavior and operational controllability.

It requires sufficient representative or production material to support conclusions about observed behavior, control performance, intervention and recovery.

The routes are not basic and advanced versions of one service, and they are not a compulsory sequence.

The appropriate route and depth are confirmed after the initial conversation.

What the Conclusions Can Support

Depending on the agreed boundaries and the quality of the material reviewed, the conclusions may:

identify conditions that should be addressed before deployment, scaling or increased autonomy;
clarify whether a proposed next step has sufficient support;
distinguish established findings from unresolved assumptions;
prioritize remediation, testing or further confirmation;
provide a bounded readiness conclusion or recommend conditions for permitted use.

The level of confidence reflects what was actually reviewed.

Unavailable or excluded areas remain unverified, and conclusions apply only to the relevant system, workflow, environment and operating context.

Decision Responsibility Remains With the System Owner

The review supports the judgment of the responsible system owner. It does not replace that judgment or transfer responsibility for the final decision.

An Independent AI Risk Review is not a formal audit, certification, legal opinion, compliance approval, penetration test or red-team exercise.

More detailed boundaries are available on the Professional Boundaries page.

Discuss Your AI System

Begin with a short overview of:

what the system or workflow does;
its current stage;
the decision, change or next step being considered;
the main uncertainty you would like to clarify.

You do not need to choose the final review route before making contact.

Detailed technical materials are not required for the first message.

Please do not submit passwords, credentials, API keys, sensitive personal data, confidential documents, unredacted logs, production traces, source files or archives in your initial email.

Discuss Your AI System