What Databricks Vibe Data Modeling Verifies, and What It Cannot

In July, Databricks published Vibe Data Modeling: a notebook that takes a plain-English description of a business and deploys a governed Silver-layer model, metric definitions included, into Unity Catalog. Their number is "a deployed Minimum Viable Model in under two hours," against the "six months to three years" that hand-built Silver projects have taken. The public repository now holds 40 industries, 80 models, 23,092 tables and 11,661 metric views, every one generated by the same agent, and over the last two weeks it gained a set of skills for fitting a generated model to a customer's real data.

I want to credit this properly before I argue with it. The Silver layer is where an analytics stack is made or broken, and the two-year modeling project is the one most companies never finish. Databricks did not stop at templates. They shipped the machine that makes them, one that produces a model in your words and checks it against 251 rules, and they put the agent itself in the public repository. That is a real advance.

It also leaves the hard problem exactly where it was. Governed, internally consistent, and true to the business are three different properties. Vibe Data Modeling delivers the first two by construction. Nothing in the pipeline tests the third, because nothing in the pipeline can see the business. Generation got automated. Verification of metric definitions did not, and verification is where the expensive mistakes live, because they do not look like mistakes.

What Databricks actually built

The repository describes an eight-stage agent that runs from tier classification through domain, table and column generation to foreign-key linking, naming, and finally metric view synthesis, with structural validators gating each stage before the next starts. The launch post puts a number on the ruleset: "The agent enforces 251 rules across 20 groups." Two reviewer personas run on top: "The Domain Architect reviews each domain in isolation; the Global Architect reviews the whole model." Residuals, which the post lists as "denormalized keys, cross-domain duplicates, unlinked or cyclic foreign keys," route to a sandboxed repair step, and the review reruns up to eight passes until clean. Every vibe produces a new version. Nothing is overwritten.

The rules themselves are worth reading. The June post that introduced the industry models lists the rule groups as they stood then: "naming conventions, semantic deduplication, foreign keys, primary keys, normalization, domain structure, data types, classification tags, relationship/DAG enforcement, quality, product design, vibe constraints, physical schema deployment, and subdomain sizing." Every group is about the model's shape, or about its conformance to what you asked for. Not one is about whether a definition is true.

The output is not just tables. It includes Unity Catalog metric views, which Databricks describes as "reusable KPI definitions on the products, ready for AI/BI dashboards and Genie." So the agent is not only shaping the warehouse. It is writing the definition of revenue.

The quality table in the repository is the part I respect most: across all 80 shipped models, zero foreign-key cycles, zero dangling keys, zero self-references on primary keys. A rule a machine enforces on every run beats a standard a document enforces never. One precision about what "every key resolves" means, though. The launch post says foreign keys "are applied as informational constraints," and Databricks' own documentation says such constraints "are informational only and aren't enforced." The keys are verified against the model at generation time. Nothing in the warehouse enforces them against the data afterward.

Databricks also makes the honest case against its own older product line: "A template is the average model for a sector; by construction it is nobody's actual business." Vibe is the answer to that, your divisions and your terminology, generated in hours. The launch post's own bullet reads: "Trustworthy by construction: 251 enforceable rules, two architect reviews, and an agentic loop that proves the model before it ships." And the repository's license says: "verify against your organisation's specific business rules and regulatory context before production use." Both are true, and they are about different things. They are telling you the gap exists. They are just not closing it.

Three things that all get called governed

Pull the word apart and it covers three claims that have nothing to do with each other.

Internally consistent. Every foreign key lands on a real table. No cycles. No duplicate names across domains. Every column typed and tagged. Vibe Data Modeling checks all of this deterministically and reports where it fails: the 40 smaller models ship with zero findings, and the 40 enterprise models ship with 34 duplicate names and 15 unlinked tables called out in the notes.

Matches what you declared. The model has the domains you asked for, in the words you used, with the metric you described. Databricks states the governing principle plainly: "what you say wins." If you say "exactly 10 domains," no tier classifier may add an eleventh.

True to the business. The metric view named revenue computes the number Finance would sign. The table named customer means what Sales means by customer. Where two teams hold two metric definitions of the same thing, the disagreement is visible instead of buried.

The first two are properties of the model measured against itself and against the prompt. The third is the model measured against the world. A validator can confirm that order.customer_id points at a real customer table. It cannot confirm that revenue booked at order creation is what your accountants recognize. None of the 251 rules can test that, and that is not a criticism of the rules. Look at the list above, and there is nowhere for such a rule to live.

Measured against, not proven by. A warehouse cannot prove a definition true. Whether revenue is net of refunds lives in accounting policy, in customer contracts, and in a controller's head, and no query returns it. What the evidence can do is show whether a definition is still supported by how the business actually operates. A refund table that nothing named revenue touches is evidence. So is a settlement date that every revenue view ignores. And it can say whether there is enough of that evidence to certify a definition as the company's current one. That is the narrower claim this piece makes, and it is enough.

Two metric definitions of revenue

Here is what the gap looks like in one metric. Describe a subscription business to the agent and ask for revenue. The model comes back with an order table that carries an amount, and a metric view that computes, in effect:

SELECT SUM(amount) AS revenue
FROM silver.sales.order
WHERE created_at BETWEEN :period_start AND :period_end

That passes every structural rule. The key resolves, the type is right, the name is unique, the view installs cleanly. It is also not how this company books revenue. Finance recognizes revenue net of refunds, at settlement rather than at order creation, and the SQL they actually run looks like this:

SELECT SUM(o.amount) - COALESCE(SUM(r.amount), 0) AS revenue
FROM silver.sales.order o
LEFT JOIN silver.sales.refund r ON r.order_id = o.id
WHERE o.settled_at BETWEEN :period_start AND :period_end

Both queries are internally consistent. Both are rule-compliant. For a quarter with 12.4M in orders created, 0.5M refunded, and 0.3M created in the quarter but settled after it (and, to keep the arithmetic simple, nothing settling late from the quarter before), the first says 12.4M and the second says 11.6M. The generated model certifies the wrong one, and nothing in the pipeline can tell, because the declaration said revenue and revenue is what it built. The refund table sits right there in the same model, with a foreign key to the order table, and no metric view named revenue references it.

Now the second failure, which is worse than the first. Sales reads the metric view. Finance keeps its own SQL. Two numbers with the same name now live in the same warehouse, one of them governed, and the governed one is the wrong one. We have written before about what happens when four teams bring four revenue numbers to the same meeting. A generated model does not prevent that. It adds a fifth number with a certificate.

"What you say wins" is the right rule for a generator

I do not think Databricks got the principle wrong. For a generator, obedience is the correct design. Nobody wants an agent that argues about how many domains you asked for.

But the principle sets the ceiling of correctness at the declaration. A system built to obey what you say will faithfully build the wrong model when what you say is wrong, and then certify it as consistent. The person writing the vibe is usually a data architect. The person who knows revenue is net of refunds is a controller who will never open the notebook. The declaration encodes what its author knew on the day it was written.

Databricks is right that a sector template is nobody's business. A model in your terminology is a real step up from that. But your terminology is not your truth. In AdventureWorks, the sample warehouse half the industry learned SQL on, an order header carries SubTotal, TaxAmt, Freight and TotalDue. Revenue is one of those, or an arithmetic of them, and nothing in the schema says which. Every structural rule passes for all four.

The skills added to the repository over the last two weeks, now consolidated under model-genie-skills, push closer to the customer's reality, and they deserve the credit. The assessment step profiles the raw layer, folds in existing production tables, and grades each target as a full, partial or blocked fit. That answers a real question: does the data exist to fill this shape? It does not answer the other one: is this shape's definition the one the business uses?

Even the mechanical layer under a governed metric carries a caveat. The documentation for querying metric views from BI tools says that for non-additive measures, "grand totals might display incorrect values because summing pre-grouped ratios is not mathematically equivalent to computing the ratio over the full data set." To be precise, that is a different failure from the one in this essay: a BI tool re-adding numbers the view computed correctly. It is worth quoting because the arithmetic layer, which is far better understood than the semantic one, still needs a warning label. Governed describes who has custody of a definition. It says nothing about whether the definition is true.

A snapshot of a business that moves

Everything so far is about the day the model is written, and that is the smaller half of the problem. A generated model is a photograph of the business as one person described it on one afternoon. Photographs do not update. The day after, the business moves, and a definition that was right when it was written starts going wrong without anything in the model changing.

Take the revenue view from above and assume it was correct on the day it deployed. Then Finance moves recognition from order creation to settlement, and the view still keys on the creation date. A usage-based tier launches, revenue now lives in two tables, and the view still sums one. Two regions merge and customer IDs get reassigned, and every foreign key still resolves. Three quarters later the model passes all 251 rules exactly as it did on day one. Not one of those changes raised a flag, because none of them is a structural change. They are changes in what the business means, and the model holds meaning as a snapshot.

Databricks knows the business moves. Every vibe is versioned, and the repository's own notes say the last part of an enterprise's shape is "a follow-up vibe-iteration the agent can take on." That is the honest answer for a generator: when the business changes, describe it again. But it depends on a person noticing that the description has gone stale, and a stale number looks exactly like a fresh one. The revenue view above returns 12.4M on the day it is wrong with the same confidence it had on the day it was right. Retrospective modeling of a fixed business is what got automated. Keeping the model true to a business that moves every day is what did not.

That is the case for an operating model of the business, not a snapshot of it: definitions bound to the data that tests them, owned by the people who hold their meaning, and checked again every time the data or the meaning moves. What makes it urgent now is who reads the definition. Databricks describes the generated metric views as "ready for AI/BI dashboards and Genie." The implementation skills go one step further: the documentation station creates "an auto-generated Genie space so the domain is queryable in natural language the moment it's built." The chat interface is not an option someone adds later. It is wired up by default. So the 12.4M revenue view is not a number one analyst might catch while writing the query. It is the number Genie returns when a sales lead types "how did revenue do last quarter," and the number a copilot drops into the board summary, and neither of them ever sees the SQL. One analyst with a stale definition gets corrected in a meeting. A metric view that a chat interface reads on behalf of the whole company gets corrected only after it has answered every question the same wrong way. An operating model of the business was worth having before. With AI reading the definitions, it is the difference between one wrong number and one wrong number in every answer.

The layer above Silver

The interesting product is not a rival generator. Generation is cheap now, and it will get cheaper. The interesting product reads a model, generated or hand-built, as its input, and does the three things no structural gate can.

First, it checks each definition against the evidence in the warehouse. A refund table exists and nothing named revenue touches it: flag. A settlement timestamp exists and every revenue view keys on creation date instead: flag. These are not style checks. They are the warehouse contradicting the declaration.

Second, it finds where metric definitions fork. Sales' revenue and Finance's revenue are both in the warehouse, and the honest behavior is to surface both with their provenance, not to pick one because it was declared louder.

Third, when it cannot verify a definition, it refuses to certify it, and the refusal carries what it found. We made the case in an earlier essay that the way to evaluate an AI data product is to ask it a question it should know not to answer. A model generator can never refuse. Its job is to produce. The layer above it has to be willing to say: this is consistent, and I cannot tell you it is right.

We call our version of that layer the Living Map. It does not generate the model. It starts from the tables already in the warehouse, and binds each metric to the measure that computes it, with the formula, the owner who answers for it, and the history of who changed it and why. Structure is re-crawled, so a renamed table surfaces as a change instead of a silent break. Meaning goes through a person, so a definition goes live when its owner blesses it, and a rescan never overwrites that verdict. And when the map cannot anchor a number, it refuses to render one. Vibe Data Modeling and a Living Map are not competing for the same job. One produces the theory of the business in an afternoon. The other keeps asking whether the theory is still true.

A generated model is a theory of the business, produced faster than ever before, and a theory needs an instrument that can tell it when it is wrong. That instrument is what a Living Map is for, and building it is the open problem. It is the one we spend our days on, and we will not claim it is solved, here or anywhere.

When a Silver layer took two years to build, the bottleneck was building it. Databricks moved that bottleneck, which is what real progress looks like. Now the model takes an afternoon, and the bottleneck is every afternoon after that: finding out which of those metric views your CFO would sign, and whether the answer is still yes.