Insurance AI Audit: 6 Critical Tests Regulators Will Apply

An insurance AI audit now starts with evidence. Regulators want to know which models an insurer uses, what risks they create, and whether controls work in daily operations.

The National Association of Insurance Commissioners has set out a practical approach. Its draft AI Risk Evaluation Supplement helps US state regulators review AI use in insurance. It covers customer impact, financial impact, governance, model records, and data.

Version 5.0 was released for comment on 31 August 2026. The comment period ended on 29 September. The draft may still change. Yet it gives insurers a clear view of the evidence that an examiner may request.

How the insurance AI audit moves from policy to proof

The NAIC adopted its Model Bulletin on the Use of Artificial Intelligence Systems by Insurers in December 2023. The bulletin expects an insurer to maintain a written AI systems programme that fits the risk of its use cases.

It also keeps the insurer responsible for systems supplied by vendors. A state department may request information during an inquiry or a market conduct exam.

The new supplement turns those broad duties into a review process. Existing handbooks still guide an examiner’s choice of firms and systems. Once a review starts, however, the supplement helps the examiner trace a path from the firm’s AI programme to selected models, data, and controls.

As a result, general claims about responsible AI will carry little weight on their own. Each answer may need a policy name, a page number, an inventory entry, or a model record.

The NAIC tested the approach in 2026 with regulators from 12 states. Feedback from regulators and insurers shaped later drafts. The name also changed from AI Systems Evaluation Tool to AI Risk Evaluation Supplement. This change makes its role within existing supervision clearer.

An insurance AI audit starts with an inventory

The first step is scoping. Exhibit A asks for details about the insurer’s AI systems and models. Version 5.0 makes the model inventory an express part of that request.

The draft separates models with a direct customer impact from models with a material financial impact. It also asks about model types. In addition, either the regulator or the insurer may set a materiality threshold. When the insurer sets it, the threshold must be disclosed.

This design reflects the range of insurance models. A claims triage model, a fraud score, a pricing model, and an internal forecast create different risks. The first inventory lets the examiner choose where to look more closely.

Older methods may still be in scope. Version 5.0 brings machine learning and generalised linear models into the initial review. After seeing the answers, the examiner can narrow the inquiry.

How an insurance AI audit assesses inherent risk

The draft first looks at inherent risk. This is the risk created by the model before controls reduce it. Therefore, a sound governance policy cannot hide the impact of a high-risk use case.

A model that may deny a claim or change a premium starts with greater risk than a tool that summarises internal notes. The same is true for a model that could affect reserves or the insurer’s financial position.

Customer impact and financial impact give examiners two useful views. Some models affect both. Others may need close review because they affect customers, even when the effect on the insurer is small.

Technical complexity is only one factor. A simple model used for millions of people may matter more than a complex tool used for a low-risk office task. Therefore, the depth of review should follow the possible harm and the scale of use.

The insurance AI audit evidence trail

The draft uses four exhibits. Together, they connect the firm’s AI programme to each model and its data.

Exhibit A: the model population

Exhibit A supports the first scope review. It records the model type, customer impact, financial importance, and inventory source. The examiner can then select a smaller sample for deeper work.

Exhibit B: the AI programme

Exhibit B reviews the insurer’s AI systems programme. Version 5.0 adds questions on explainability, transparency, materiality, and third-party models. The checklist may require a document name and page number for each answer. Consequently, a general statement must be tied to proof.

Exhibit C: the selected model

Exhibit C examines one model in detail. It asks about the purpose, model type, development method, risks, and limits. The draft also invites a rating such as high, moderate, or low. Thus, the model record should explain how the model works, where it may fail, and how serious that failure could be.

Exhibit D: the data

Exhibit D follows the data used by the model. Version 5.0 links each dataset to the models that depend on it. This link matters because outside data and proxy fields can shape an insurance decision. Weak or biased input data can damage even a well-built model.

What an insurance AI audit is likely to test

An insurer should be ready to produce a current inventory. It should also explain why each model is inside or outside the review scope.

Next, the examiner may compare written duties with real decision rights. Board and management roles should match the way the firm approves, changes, monitors, and stops a model. Validation reports and change logs should support those claims.

For a customer-facing model, the insurer should be able to explain an outcome. It should show how a customer can challenge that result. In addition, vendor oversight should reach the model and data records needed for supervision.

Human review will receive close attention. A reviewer needs enough facts to question an output. The reviewer also needs the authority and time to intervene. Therefore, the insurer should record when review occurs, what the reviewer sees, and why an override was made.

Generative AI creates an extra test because the same request can produce different answers. FinTech Central’s analysis of banking AI reliability shows why repeat testing matters. Evidence may need to cover prompts, settings, retrieval sources, test cases, limits, and action taken after a poor result.

How an insurance AI audit treats vendor models

Many insurers buy models, data, and cloud services from outside firms. FinTech Central’s guide to AI in health insurance shows how these links span underwriting, claims, and third-party administration.

The NAIC Model Bulletin keeps the insurer responsible for regulated decisions. Version 5.0 supports that rule through a direct question on third-party model oversight.

Procurement teams therefore need audit rights, document rights, and access to performance and change data. A claim of trade secrecy will not answer a concern about unfair bias, poor data, or an unexplained adverse result.

The insurer still needs enough evidence to test the use case, monitor outcomes, review complaints, and stop the model. Controlled knowledge can also help staff use approved rules, as explained in FinTech Central’s financial services RAG framework.

What an insurance AI audit means for Indian insurers

The NAIC supplement is a US tool. Its value for India lies in the method. Indian insurers already use AI in sales, underwriting, claims, fraud checks, and service. They also depend on many technology and data partners.

IRDAI may use a different legal structure. Still, supervisors will ask similar questions about active models and the decisions they affect. They will also review the data used and the tests for bias or error. Finally, they will want to know who can step in when a customer is harmed.

Indian insurers should build the evidence now. A group-wide inventory should link each system to its owner, purpose, model type, data, vendor, affected customers, financial importance, validation status, monitoring measures, and retirement plan.

Teams should create these records during development and use. Customer-facing models should also link to complaints, overrides, and outcome tests. This evidence lets the board and the regulator compare policy with real results.

Six insurance AI audit actions for insurers

1. Build one inventory for in-house models, vendor models, embedded AI features, and material data services.
2. Rate each use case for customer impact and financial impact. Record the threshold and the reason.
3. Link each governance answer to a policy, committee record, test report, monitoring result, or change log.
4. Maintain a model record that covers purpose, method, known risks, limits, data needs, and approved uses.
5. Test whether human reviewers can challenge outputs. Study the causes and results of overrides.
6. Review vendor contracts for audit, document, change notice, and suspension rights.

The insurance AI audit shifts from policy to proof

The NAIC draft may change after consultation. Its direction is already clear. Supervisors want firms to know which models they run, understand the risk of each use, and prove that controls reach the model and its data.

Many firms have sound principles but scattered evidence. The best-prepared insurers will connect inventory and materiality to model records, data sources, validation, monitoring, human action, and customer redress. A long AI policy cannot replace that chain of proof.

Sources

• NAIC Big Data and Artificial Intelligence Working Group: AI Risk Evaluation Supplement version 5.0 exposure notice
• NAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers
• NAIC artificial intelligence insurance topic page