AI Governance Starts with a System Inventory

AI governance starts with an inventory. A policy cannot manage systems, models, prompts, data flows, or decisions that an organisation has not identified. The first useful control is a current register of where AI is used and who is accountable for it.

NIST’s AI Risk Management Framework uses Govern, Map, Measure, and Manage as connected functions. An inventory gives those functions something concrete to govern and review.

On this page

• What an AI inventory should contain
• Why the workflow matters more than the model label
• How to classify AI use cases
• What evidence belongs beside each entry
• How to measure AI performance and risk
• How to govern third-party AI services
• What to do when the inventory is incomplete
• What does not matter as much as the inventory
• Turn the design into an operating control
• Test the failure path
• Measure the result without false precision
• Review change and ownership
• Keep the handoff explicit

What an AI inventory should contain

An inventory should describe the use case in plain language, not merely name a vendor or model. Record the business process, users, inputs, outputs, decision role, affected people, owner, and operating environment.

Also record whether the system drafts, recommends, ranks, classifies, predicts, or makes a decision. The distinction changes the review needed and the evidence an operator must retain.

• Use-case name and purpose.
• Model or service, version, and provider.
• Data types, users, owner, review date, and status.

Why the workflow matters more than the model label

Two teams can use the same model for very different risks. A tool that summarises internal notes is not the same as one that influences access to housing, employment, credit, healthcare, or education.

Map the surrounding workflow. Identify the human review step, the system of record, the escalation route, and what happens when the output is wrong or unavailable.

• Name the decision or task.
• Record human review and override.
• Define the failure and escalation path.

How to classify AI use cases

Classification should help people act. Use simple fields such as impact, sensitivity, autonomy, scale, affected groups, and reversibility. Avoid a complicated score that no owner understands.

A low-risk drafting assistant may need basic data and access controls. A system affecting people’s opportunities may need deeper testing, documentation, monitoring, and approval.

• Impact on people and services.
• Sensitivity of input and output data.
• Autonomy, scale, and reversibility.

What evidence belongs beside each entry

Evidence turns an inventory from a spreadsheet into an operating control. Link the intended-use statement, data assessment, test results, approval, user guidance, incident history, and monitoring record.

Keep evidence versioned. A model update, prompt change, new data source, or workflow change can alter the risk even when the product name stays the same.

• Test cases and limitations.
• Approval and accountable owner.
• Change history and monitoring results.

How to measure AI performance and risk

Measure the process, not only model accuracy. Track correction rate, refusal rate, escalation volume, latency, data-quality issues, user reliance, and outcomes that matter to the workflow.

A result can look accurate in a sample and still fail in production because the input distribution changed. Define the review period and the threshold that triggers action.

• Baseline before deployment.
• Representative and edge-case tests.
• Thresholds, alerts, and corrective actions.

How to govern third-party AI services

A hosted service does not remove accountability. Review the provider’s data handling, retention, access controls, audit information, service limits, change process, and exit options.

Do not accept a generic promise as the whole assessment. Tie each important requirement to a contract term, a technical setting, or evidence that can be checked.

• Data use and retention terms.
• Administrative and audit controls.
• Exit, portability, and incident duties.

What to do when the inventory is incomplete

Start with the systems most likely to create material impact: customer-facing decisions, sensitive data, privileged operations, and high-volume workflows. Publish the inventory owner and a deadline for the next review.

An incomplete register is still useful if it is labelled as incomplete. Hidden use is the larger risk because controls cannot be assigned to an unknown system.

• Prioritise high-impact uses.
• Mark unknowns rather than guessing.
• Set an owner and review date.

What does not matter as much as the inventory

A polished AI policy, a long vendor questionnaire, or a list of approved tools does not show how AI works inside the business. Governance becomes practical only when it is attached to a real workflow and current evidence.

Keep the register readable enough for operators, risk owners, procurement, and leadership to use together.

• Do not confuse a policy with implementation.
• Do not count pilots as governed production.
• Do not hide model changes behind a stable brand name.

Turn the design into an operating control

A design becomes an operating control when a named person can perform it, another person can review it, and the organisation can show evidence that it happened. Write the trigger, the action, the expected result, and the exception path in language an operator can use during a busy day.

Keep the control close to the workflow. If staff must leave one system, search an unrelated document, and ask another team before acting, the control will be skipped when pressure rises. Reduce that friction without hiding the decision.

• Name the trigger and operator.
• State the expected result.
• Record the exception and escalation.

Test the failure path

Happy-path demonstrations are useful for learning, but they do not prove resilience or security. Test incomplete data, unavailable dependencies, expired credentials, unexpected volume, delayed input, and a human decision that disagrees with the system output.

A failed test is useful when it produces an owner, a correction, a retest date, and a decision about whether the remaining risk is acceptable. Do not quietly convert a failed test into a passing narrative.

• Choose realistic failure cases.
• Record evidence and observed impact.
• Assign correction and retest dates.

Measure the result without false precision

Choose a small set of measures that show whether the control or workflow is working. Define the denominator, time period, data source, owner, and action that follows a meaningful change.

Use estimates and scenarios honestly. A precise-looking number built on incomplete data is less useful than a range with a clear boundary and a plan to improve measurement.

• Keep definitions stable.
• Separate measured, estimated, and projected results.
• Connect each measure to a decision.

Review change and ownership

Technology environments change through releases, suppliers, data, policies, identities, and user behaviour. A control that was adequate at launch may not remain adequate after a material change.

Set a review trigger as well as a calendar review. When the owner, dependency, data, exposure, or failure mode changes, revisit the design and keep the decision record with the evidence. Keep the next review date visible.

• Record version and change.
• Review after material events.
• Keep owner, date, and decision visible.

Keep the handoff explicit

Most operational failures occur between teams, systems, or stages of work. State what one owner must provide, what the next owner checks, and what happens when the handoff is late, incomplete, or rejected.

This simple contract improves incident response and day-to-day work. It also makes automation safer because the input, output, and exception are visible rather than implied.

• Name the sender and receiver.
• Define the input and acceptance check.
• Record rejection, retry, and escalation.

Operating rule: Name the owner, the evidence, and the action before calling a technology control complete.

Comparison table

FAQ

Is an AI inventory only for large companies?

No. A small register is useful for any organisation using AI with customer data, sensitive information, or business decisions.

Should every AI tool need the same approval?

No. Approval should match impact, sensitivity, autonomy, scale, and reversibility. A drafting aid and a high-impact decision system should not follow an identical path.

How often should an AI inventory be reviewed?

Review it on a defined schedule and after material changes to the model, data, workflow, provider, or affected population.

What is the first field to add?

Start with the use case, owner, purpose, inputs, outputs, and decision role. Those fields reveal what deeper controls are needed.

How can a team start without rebuilding everything?

Start with one important workflow, define the owner and evidence, test the failure path, and expand only after the operating result is understood.

What should be recorded after a review?

Record the scope, date, evidence, decision, owner, unresolved risk, and next review or correction. A short honest record is more useful than an impressive but untraceable claim.

When should the design change?

Change it when the workflow, data, identity, dependency, supplier, exposure, user group, or failure mode changes materially. A calendar review alone may miss the event that changed the risk.

What is a useful first metric?

Choose a measure close to an operating decision, define its denominator and time period, and state what action follows when it crosses the agreed threshold.

Conclusion

The useful technology decision is the one that can be tested. Define the operating problem, record the evidence, assign ownership, and review the result after launch. Clear scope beats a large claim, and a measured workflow beats a polished demo.

Sources

• NIST AI Risk Management Framework
• NIST AI RMF 1.0 PDF