AI for insurance
In insurance, one clause separates covered from not covered. We build retrieval over policy wordings, endorsements and riders, with a mandatory clause citation, and we measure whether the system finds it before the answer reaches a policyholder.
Key takeaways
An insurance assistant that paraphrases coverage without citing the clause is not useful: nobody can verify it, and a wrong answer about a covered peril ends up as a complaint.
Policy wordings break generic systems for a concrete reason: exclusions are written as negations, and "the policy covers X" and "the policy does not cover X" sit almost on top of each other in vector space.
We measure recall@k at clause level, not document level. Finding the right wording is worthless if the answer comes from the wrong article inside it.
The rule that is not negotiable: when retrieval comes back empty or ambiguous, the system says so and hands off to an adviser. In insurance, guessing is worse than not answering.
What people actually ask an insurer
These are not open questions. They are specific queries against a document the policyholder owns but cannot read, and every one of them is answered by a concrete clause.
- The most frequent question and the most dangerous. It depends on the cover purchased, on the endorsements and on the exclusions — which usually live in three different parts of the same document.
- It varies by cover and sometimes by event. A wrong number here turns into a formal complaint.
- A contractual deadline with a direct consequence: past the term, the right is lost.
- A list that changes by line and by cause. Answering it correctly cuts incomplete filings, which are the largest hidden cost in a claims department.
- Liability, undeclared drivers, beneficiaries. It depends on the specific schedule, not the general wording.
Why a policy wording breaks a generic RAG
A policy wording is not an ordinary document. It is a three-layer structure — general wording, specific schedule, and endorsements or riders — where the lower layer modifies the one above it. The same article number means different things across two products from the same carrier, and an endorsement issued six months ago may have changed the cover the general wording still describes.
The first failure to show up is exclusions. They are written as negations — "is not covered", "are excluded", "unless" — and in vector space "the policy covers water damage" and "the policy does not cover water damage" are almost the same point. A system relying on semantic similarity alone returns the exclusion when asked about the cover, and the other way round. It is the same entity-collision problem we solved running a RAG for two million users: you fix it by extracting structured metadata at ingestion — line, product, effective dates, clause type, polarity — and filtering before ranking, not by trusting the embedding.
The second decision that changes everything is the retrieval unit. If the system retrieves documents, it finds the right wording and fails inside it. You have to chunk by clause, keep the hierarchy — chapter, article, paragraph — and make the answer carry that reference all the way to the screen, so an adviser can open the document and verify in one click.
The numbers we ask for before it talks to a policyholder
These are the metrics of the evaluation set, built from real questions off the service channel with the correct clause validated by the carrier’s technical team.
| What is measured | What it means | What happens if it fails |
|---|---|---|
| Clause-level recall@k | Across the evaluation set, how often the correct clause appears among the k retrieved. At clause level, not document level. | Chunking gets revisited and hybrid search goes in: insurance vocabulary has exact terms that embeddings alone do not separate. |
| Groundedness | Whether every statement in the answer holds up against the cited clause, adding nothing that is not there. | The instruction gets tightened and post-generation verification goes in: an answer that does not hold up against the passage does not ship. |
| Exclusions handled correctly | A subset of the evaluation set is questions whose correct answer is "not covered". Measured separately because they fail most often. | Clause polarity gets labelled at ingestion and used as a filter before ranking. |
| Correct refusal rate | When the policy says nothing about what was asked, the system has to say so and escalate rather than approximate. | The confidence threshold drops and it hands off to an adviser. We would rather over-escalate than under-answer. |
The evaluation set ships in an open format and belongs to the carrier: it verifies the system without us, and it grades any other vendor by the same yardstick.
From the wording to production
- Document inventory and reference setWe map which documents exist per line and product, in which versions, and who maintains them. In parallel we build 80 to 150 real questions off the service channel with the correct clause validated by your technical team.
- Ingestion with metadata, not just textClause-level chunking that keeps the hierarchy, plus extraction of line, product, effective dates, clause type and polarity. That is what later makes filtering before ranking possible.
- Measure before you open itWe run the reference set and publish the number. If clause-level recall@k does not reach the agreed threshold, it does not go to production: chunking and retrieval get adjusted until it does.
- Internal channel firstIt launches with advisers, not policyholders. They see the cited clause and report what fails, and that feeds the evaluation set through the first weeks.
If the carrier has three products and one twenty-page wording that changes once a year, this is over-engineering. A well-built FAQ and a search box over the PDF solve 90% of it at a fraction of the cost.
The conversation changes when there are several lines, dozens of live products, endorsements that modify cover, and different versions coexisting — that is, when finding the right clause is already a job in itself. That is where measurement stops being a luxury.
Related services
RAG systems
Chunking, reranking and hybrid search, evaluated with recall@k and NDCG.
View serviceProject rescue
The system is already in production and answers badly. We measure it and determine what to fix.
View serviceAI chatbots and assistants
With the evaluation set run before it ever reaches production.
View serviceAI consulting
Architecture, evaluation criteria and cost per query before writing code.
View serviceMore from the blog
Frequently asked questions
Can the assistant tell a policyholder whether their claim is covered?
It can show them what their policy says and cite the clause, which is a different thing. Deciding coverage on a specific claim is a technical act by the carrier, involving facts the system does not have: cause, adjuster findings, account status. We design it to answer "under article 4.2 of your schedule, this cover includes X with a deductible of Y" and to escalate the moment the question moves from what the policy says to what will happen with the claim. That boundary gets defined in writing before anything is built.
How do you stop the system confusing a cover with an exclusion?
Two things, and neither is the model. First, labelling the polarity of every passage at ingestion — cover, exclusion, condition or definition — and filtering on that label before ranking. Second, measuring it separately: the evaluation set contains a subset of questions whose correct answer is "not covered", and its accuracy is reported apart from the headline number, because the average hides it.
What happens when a wording changes or an endorsement is issued?
The new version gets reindexed with its effective dates, and retrieval filters by the date of the claim or the query rather than by "the latest". That is the part almost no vendor implements and the cause of the most expensive failure: answering with today’s wording about a policy issued two years ago under different text. Effective dates are mandatory metadata at ingestion, not a detail.
Is this for a broker or only for a carrier?
It actually fits a broker better. A broker handles products from several carriers, and their advisers need to compare cover across different wordings — exactly the work a good retrieval system does well and a person does slowly. The difference is that in a brokerage the sources are third-party documents, so how they are obtained and kept current has to be solved first.
Does our policy data leave our infrastructure?
Only if you accept that. When the case requires it we deploy on Azure OpenAI or Amazon Bedrock inside your own subscription, so documents and queries never leave your cloud. In any configuration we use enterprise plans where data sent through the API is not used for training, and we filter personal data before anything reaches the model.
Does it integrate with the policy administration system?
Yes, and it is usually what separates a useful assistant from a demo. Questions like "what is my deductible?" need that policyholder’s actual policy, not the generic wording. We integrate over API against your core system or CRM to pull the in-force policy and its endorsements, and retrieval then runs over the document that applies to that person. Without that integration the assistant can only talk about products in general.
How long does it take and how is it priced?
A first line with orderly documentation usually takes six to eight weeks to production, with something running from week one. Subsequent lines go faster because ingestion and measurement are already built. It is quoted with fixed scope, price and date in a proposal 48 hours after the first call, and the price is driven by document volume, how many systems have to be integrated, and what precision threshold the case demands.
We already have an insurance chatbot and it answers badly. Do you fix it?
That is the most common case we get. Start with the diagnostic rather than a rebuild: in one week we measure the existing system against a reference set of your own and hand back the failures ranked by impact, each with its fix and estimated effort. The model is almost never the problem — chunking of the wording and missing filters by product and effective date usually are, and that gets rebuilt without throwing away the rest. If the diagnostic does not reach three actionable findings, it is not billed.
Book 15 minutes
Tell us what you are building, or what stopped working. You leave the call with a concrete answer: it can be fixed, it can be built, or it isn't worth it.