One of the most useful things we can tell an architect evaluating AI-assisted modeling is that, inconsistency is the reality of every LLM-based tool. The question that matters isn’t whether generation is probabilistic, it’s whether the workflow lets you catch the errors before they reach your model. Here, review isn’t a safety net bolted onto generation - it’s the operating model.
There are two ways to build a data platform model. Source-driven starts from the systems you have; model-driven describes the business first and derives the model from it. The latter is the tougher goal to achieve, whereby AI-assisted derivation is the current attempt. The promise is always the same - capture business concepts, run the derivation, get Data Vault structures - but behind it sits everything from a trivial one-concept-one-hub mapping to a real derivation that decides roles, grain, and historization. A textbook product demo can’t tell the two apart, so we built a scenario designed to fail.
An insurance data model built to break AI derivation
7 Business Concepts describe a deliberately simplified insurance business:
- Party (PartyId, Name, DateOfBirth, Address)
- Policy (PolicyNumber, StartDate, EndDate, Status, PaymentFrequency)
- Product (ProductCode, Name, LineOfBusiness)
- Coverage (CoverageId, SumInsured, Deductible, StartDate)
- Claim (ClaimNumber, EventDate, ReportedDate, Status, ClaimType)
- ClaimPayment (PaymentId, Amount, PaymentDate, PaymentMethod)
- Agent (AgentNumber, Name, Agency).
8 directed connections between them, carrying several different traps for automated derivation:
- Party holds a policy, is insured under a policy, and benefits from a policy — the same pair related three times with three different meanings. A naive derivation merges them into one relationship, wrong in a way no one notices until the first beneficiary report.
- A claim is reported against a coverage, not a policy. There’s deliberately no Claim–Policy connection, so a shortcut to Policy gets the grain wrong.
- Claim payments are pure transaction records with no lifecycle of their own; the derivation has to choose between an independent entity and transactional detail.
- An agent manages policies, and that assignment changes over time — a history requirement a plain relationship can’t satisfy.
Every attribute was classified up front (business key, attribute, measure, date), so keys were declared, not guessed: this test is about structural decisions, not key detection.
How AI-assisted Data Vault derivation works
If you skipped part 1 of this series of articles: Deriving Data Models from Business Concepts in biGENIUS-X, Convert to Model Objects derives model objects for a chosen target (Raw Vault, Business Vault, or Data Mart) from the concepts, steered by an optional free-text prompt, and gives every proposed object a written justification next to a preview diagram. You apply the proposal whole, or refine the prompt and derive again.
This article is about that derive-and-review loop; source mapping (manual today) and code generation come after.
Writing the derivation prompt: roles, grain, and history
The concept graph carries the structure and, through classification, the keys. The prompt carries what neither can: which connection is which role, the grain rules, and the history requirement.
Party appears in three different roles: policyholder (connection 1), insured person (connection 2), and beneficiary (connection 3). These are distinct business relationships and must stay separate — do not merge them into one. Claims are always tracked per coverage, never directly per policy. Claim payments are pure transaction records belonging to a claim; they have no independent lifecycle. The agent on a policy can change over time, and we need that history. Policy number, coverage ID, and claim number are stable business keys; product codes are unique across all lines of business.
Six sentences, no hubs or satellites or effectivity — a domain expert describing how the business works, pointing at connections by number where the graph’s default names can’t carry the meaning.
Run 1: correct role links, two Data Vault errors
The first run got the hard parts right: three separately named links between Hub_Party and Hub_Policy (policyholder, insured person, beneficiary), each with its role stated; the claim grain on Coverage with no Claim–Policy shortcut; every hub on the declared business keys. It also produced two real errors.
First, it dissolved Claim entirely: no Hub_Claim, just a link off Hub_Coverage with ClaimNumber as a degenerate key, then attached ClaimPayment as a further link whose justification referenced a Hub_Claim that was never generated. A link on a link is invalid Data Vault: the reasoning contradicted the structure it justified, and the preview diagram showed it at a glance. Second, the justification for Link_Agent_Policy acknowledged that “agent assignment can change over time,” then generated a plain link with no effectivity tracking - the requirement understood but not materialized.
Both errors were visible in Review & Confirm before anything was applied. A one-click-apply workflow would have shipped them.
Run 1, full model: three separately named role links between Hub_Party and Hub_Policy.
Run 2: the same prompt produces a different Raw Vault model
Before correcting anything, we ran the same prompt as before again.
The structural error corrected itself: Hub_Claim now existed, properly satellited and linked to Coverage. But ClaimPayment swung the other way. Run 1 had over-applied the “no lifecycle” sentence and dissolved Claim; Run 2 ignored it and gave ClaimPayment a full hub, in a justification that quotes “pure transaction record” and then assigns a hub anyway.
The agent history was still a plain link, but the justification now claimed “this allows agent assignment history to be tracked” — which a bare link can’t do. Run 1 missed the requirement silently; Run 2 asserted it was handled, the more dangerous failure, and exactly what review exists to catch. Even the naming shifted: Link_Party_Policyholder became Link_PolicyholderParty_Policy. Two runs, two models, each defensible in parts and wrong in different places.
The same input, two derivations. Full models above, claim chains enlarged below: Run 1 dissolves Claim into a link chain, Run 2 gives ClaimPayment its own hub.
How to review an AI-generated Data Vault model
Review works on two artifacts at once: the justification text and the preview diagram. It's faster to check the diagram than text, and because concepts, derivation, and review live in one tool, there’s no export-and-rebuild step in between. Three checks, learnable in an afternoon:
- Does the justification match the structure? (Run 1 referenced a hub that didn’t exist.)
- Does the structure match the stated semantics? (Run 2’s hub contradicted the prompt it quoted.)
- Is a claimed capability actually built? (Both runs’ agent links.)
They work because you’re comparing stated intent against structure, not reverse-engineering it.
Run 3: correcting the prompt for claim and agent history
When review finds problems, the loop is to go back, sharpen the prompt, and derive again. For run 3 we made two targeted corrections, simulating a correction round where a data architect would step in:
Claim is an independent business object with its own lifecycle and its own stable identifier (ClaimNumber). Claim payments are pure transaction records belonging to a claim; they have no independent lifecycle and must not be modeled as an independent entity with its own hub — model them as transactional detail attached to the claim. The agent on a policy can change over time, and past agent assignments must remain reconstructable — the agent–policy relationship needs explicit historization (effectivity).
Both landed with no regressions. ClaimPayment became a transactional link with a link satellite, no hub. The agent relationship came back as a driving key link with an effectivity link satellite — the standard Data Vault pattern for exactly this: one current agent per policy, full assignment history preserved, no remodeling when agents change. The derivation didn’t just accept the correction; it answered with the pattern an experienced modeler would choose, and one no concept-to-hub mapping produces.
Run 3, full model: the agent relationship as a driving key link with an effectivity link satellite.
Three additional notes:
- The naming changed a third time (Link_Party_Policy_Policyholder, arguably the best);
- The payment link’s justification never says how a payment is identified; and
- One error was ours — we drew the policyholder connection as 1:1 instead of 1:N, and the derivation faithfully reproduced it. Raw Vault forgave it because a link is always m:n-capable; a dimensional target wouldn’t have.
What three runs reveal about AI model derivation
Two things we didn’t expect: First, reviewing wasn’t about Data Vault mechanics at all — keys, hubs, and satellites were right every run. What the derivation got wrong, in both directions, was business meaning: how much lifecycle a claim payment has, whether an acknowledged history requirement deserves a structure. Even the invalid link-on-link was a semantic mistake first — Claim’s lifecycle was dissolved, and the structure followed. The errors lived exactly where the value lives. Second, the naming never settled — three runs, three conventions, all defensible. Cosmetic for a one-shot derivation, but exactly the kind of thing that needs pinning down the day regeneration ships.
From Raw Vault to star schema with the same concepts
Everything above targets Raw Vault. Switch to Data Mart and the same seven concepts yield a star schema, where the parked traps come due: the three party relationships become role-playing dimensions, ClaimPayment becomes the transaction fact, the date attributes become date roles, Agent→Agency becomes a dimension hierarchy. Cardinality — cosmetic in Raw Vault because a link is always m:n-capable — becomes structural: the N:M beneficiary relationship forces a bridge-or-simplify decision. The Business Concept layer doesn’t care; it’s architecture-neutral, not a Data Vault front end.
Limitations of AI-assisted data modeling today
Three runs bought a validated target model for one careful concept graph, one honest prompt, and two review passes. The effort moved to the front: describing the business cleanly and reviewing attentively. Two boundaries remain: source mappings are still manual — the derivation designs the target model, it doesn’t connect your staging tables; and every run creates new model objects, so “can I regenerate without losing manual work?” currently has an honest no; a prompt-based approach for it is in early development.
What the stress test leaves standing is the claim that matters: one business description, multiple technical implementations, every decision justified and reviewable before it lands. Probabilistic generation doesn’t undermine that — the review step is what makes it hold. Once the mappings are done, the model compiles to executable code for the target platform, a complete dbt project for instance, with every decision on the record.
