Generation Got Automated. Verification Didn't.

In July, Databricks published Vibe Data Modeling: a notebook that takes a plain-English description of a business and deploys a governed Silver-layer model, metric definitions included, into Unity Catalog. Their number is "a deployed Minimum Viable Model in under two hours," against the "six months to three years" that hand-built Silver projects have taken. The public repository now holds 40 industries, 80 models, 23,092 tables and 11,661 metric views, every one generated by the same agent, and last week it gained a set of skills for fitting a generated model to a customer's real data.

I want to credit this properly before I argue with it. The Silver layer is where an analytics stack is made or broken, and the two-year modeling project is the one most companies never finish. Databricks did not stop at templates. They shipped the machine that makes them, one that produces a model in your words and checks it against 251 rules, and they put the agent itself in the public repository. That is a real advance.

It also leaves the hard problem exactly where it was. Governed, internally consistent, and true to the business are three different properties. Vibe delivers the first two by construction. Nothing in the pipeline tests the third, because nothing in the pipeline can see the business. Generation got automated. Verification of metric definitions did not, and verification is where the expensive mistakes live, because they do not look like mistakes.

What Databricks actually built

The repository describes an eight-stage agent: tier classification, domain generation, sub-domain expansion, product (table) generation, attribute (column) generation, foreign-key linking, semantic dedup and naming, then metric view and ontology synthesis. Each stage is gated by structural validators before the next one starts. The launch post puts a number on the ruleset: "The agent enforces 251 rules across 20 groups." Two reviewer personas run on top: "The Domain Architect reviews each domain in isolation; the Global Architect reviews the whole model." Residuals, which the post lists as "denormalized keys, cross-domain duplicates, unlinked or cyclic foreign keys," route to a sandboxed repair step, and the review reruns up to eight passes until clean. Every vibe produces a new version. Nothing is overwritten.

The rules themselves are worth reading. The June post that introduced the industry models lists the rule groups as they stood then: "naming conventions, semantic deduplication, foreign keys, primary keys, normalization, domain structure, data types, classification tags, relationship/DAG enforcement, quality, product design, vibe constraints, physical schema deployment, and subdomain sizing." Every group is about the model's shape, or about its conformance to what you asked for. Not one is about whether a definition is true.

The output is not just tables. It includes Unity Catalog metric views, which Databricks describes as "reusable KPI definitions on the products, ready for AI/BI dashboards and Genie." So the agent is not only shaping the warehouse. It is writing the definition of revenue.

The quality table in the repository is the part I respect most. Across all 80 shipped models: zero foreign-key cycles, zero bidirectional pairs, zero dangling keys, zero self-references on primary keys. A rule a machine enforces on every run beats a modeling standard a document enforces never, and most hand-built Silver layers never get this level of structural discipline.

Databricks also makes the honest case against its own older product line. "A template is the average model for a sector; by construction it is nobody's actual business." Vibe is the answer to that: your divisions, your domains, your terminology, generated in hours. And the repository's license says something worth holding onto: "verify against your organisation's specific business rules and regulatory context before production use." They are telling you the gap exists. They are just not closing it.

Three things that all get called governed

Pull the word apart and it covers three claims that have nothing to do with each other.

Internally consistent. Every foreign key lands on a real table. No cycles. No duplicate names across domains. Every column typed and tagged. Vibe checks this exhaustively, and the checks are deterministic.

Matches what you declared. The model has the domains you asked for, in the words you used, with the metric you described. Databricks states the governing principle plainly: "what you say wins." If you say "exactly 10 domains," no tier classifier may add an eleventh.

True to the business. The metric view named revenue computes the number Finance would sign. The table named customer means what Sales means by customer. Where two teams hold two metric definitions of the same thing, the disagreement is visible instead of buried.

The first two are properties of the model measured against itself and against the prompt. The third is the model measured against the world. None of the 251 rules can test it, and that is not a criticism of the rules; look at the list above, and there is nowhere for such a rule to live. A validator can confirm that order.customer_id points at a real customer table. It cannot confirm that revenue booked at order creation is what your accountants recognize.

Two metric definitions of revenue

Here is what the gap looks like in one metric. Describe a subscription business to the agent and ask for revenue. The model comes back with an order table that carries an amount, and a metric view that computes, in effect:

SELECT SUM(amount) AS revenue
FROM silver.sales.order
WHERE created_at BETWEEN :period_start AND :period_end

That passes every structural rule. The key resolves, the type is right, the name is unique, the view installs cleanly. It is also not how this company books revenue. Finance recognizes revenue net of refunds, at settlement rather than at order creation, and the SQL they actually run looks like this:

SELECT SUM(o.amount) - COALESCE(SUM(r.amount), 0) AS revenue
FROM silver.sales.order o
LEFT JOIN silver.sales.refund r ON r.order_id = o.id
WHERE o.settled_at BETWEEN :period_start AND :period_end

Both queries are internally consistent. Both are rule-compliant. For a quarter with 12.4M in orders created, 0.5M refunded, and 0.3M created in the quarter but settled after it (and, to keep the arithmetic simple, nothing settling late from the quarter before), the first says 12.4M and the second says 11.6M. The generated model certifies the wrong one, and nothing in the pipeline can tell, because the declaration said revenue and revenue is what it built. The refund table sits right there in the same model, joined to nothing that matters.

Now the second failure, which is worse than the first. Sales reads the metric view. Finance keeps its own SQL. Two numbers with the same name now live in the same warehouse, one of them governed, and the governed one is the wrong one. We have written before about what happens when four teams bring four revenue numbers to the same meeting. A generated model does not prevent that. It adds a fifth number with a certificate.

"What you say wins" is the right rule for a generator

I do not think Databricks got the principle wrong. For a generator, obedience is the correct design. Nobody wants an agent that argues about how many domains you asked for.

But the principle sets the ceiling of correctness at the declaration. A system built to obey what you say will faithfully build the wrong model when what you say is wrong, and then certify it as consistent. The person writing the vibe is usually a data architect. The person who knows revenue is net of refunds is a controller who will never open the notebook. The declaration encodes what its author knew on the day it was written, and the day after, the business moves. Pricing adds a usage tier. A reseller channel starts booking through a different table. The declaration is stale, the model is still consistent, and consistency is the only thing being checked.

Databricks is right that a sector template is nobody's business. A model in your terminology is a real step up from that. But your terminology is not your truth. In AdventureWorks, the sample warehouse half the industry learned SQL on, an order header carries SubTotal, TaxAmt, Freight and TotalDue. Revenue is one of those, or an arithmetic of them, and nothing in the schema says which. Every structural rule passes for all four.

The skills added to the repository this month push closer to the customer's reality, and they deserve the credit. The assessment step profiles the raw layer, folds in existing production tables, and grades each target as a full, partial or blocked fit. That answers a real question: does the data exist to fill this shape? It does not answer the other one: is this shape's definition the one the business uses?

Even Databricks' own governed metric layer draws this line for you. The documentation for querying metric views from BI tools says that for non-additive measures, "grand totals might display incorrect values because summing pre-grouped ratios is not mathematically equivalent to computing the ratio over the full data set." That is a vendor stating, correctly, that a governed metric can return a wrong number. Governed describes who has custody of a definition. It says nothing about whether the definition is true.

The layer above Silver

The interesting product is not a rival generator. Generation is cheap now, and it will get cheaper. The interesting product reads a model, generated or hand-built, as its input, and does the three things no structural gate can.

First, it checks each definition against the evidence in the warehouse. A refund table exists and nothing named revenue touches it: flag. A settlement timestamp exists and every revenue view keys on creation date instead: flag. These are not style checks. They are the warehouse contradicting the declaration.

Second, it finds where metric definitions fork. Sales' revenue and Finance's revenue are both in the warehouse, and the honest behavior is to surface both with their provenance, not to pick one because it was declared louder.

Third, when it cannot verify a definition, it refuses to certify it, and the refusal carries what it found. We argued last week that the way to evaluate an AI data product is to ask it a question it should know not to answer. A model generator can never refuse. Its job is to produce. The layer above it has to be willing to say: this is consistent, and I cannot tell you it is right.

We call our version of that layer the Living Map. It does not generate the model. It starts from the tables already in the warehouse, and binds each metric to the measure that computes it, with the formula, the owner who answers for it, and the history of who changed it and why. Structure is re-crawled, so a renamed table surfaces as a change instead of a silent break. Meaning goes through a person, so a definition goes live when its owner blesses it, and a rescan never overwrites that verdict. And when the map cannot anchor a number, it refuses to render one. Vibe and a Living Map are not competing for the same job. One produces the theory of the business in an afternoon. The other keeps asking whether the theory is still true.

A generated model is a theory of the business, produced faster than ever before, and a theory needs an instrument that can tell it when it is wrong. That instrument is what a Living Map is for, and building it is the open problem. It is the one we spend our days on, and I will not claim it is solved, by us or by anyone.

When a Silver layer took two years to build, the bottleneck was building it. Databricks moved that bottleneck, which is what real progress looks like. Now the model takes an afternoon, and the bottleneck is the afternoon after: finding out which of those metric views your CFO would sign.