The corpus is the economy
Over the last month I’ve done 60+ calls with people up and down the data supply chain: brokers, wholesalers, researchers, frontier lab buyers, RL and data labs leaders, etc.
For those of you who are so inclined, I’ve also made it available as a deck.
TLDR
-
Today’s data market is immature: no units, no prices, valued only at sale.
-
Value is in moving up the chain by transforming data and getting closer to customer value, not selling raw data.
-
The current regime won’t last, but the market will. More people will train more models forever.
-
Long term it splits: a synthetic Walmart of tokens and a scarce market for novelty.
-
Novelty never runs out, both because the diffusion timeline is incredibly long and because deploying AI creates new edge cases. The firm becomes a task factory.
-
As data gets priced, it enters the capital structure, and data allocation becomes a core CEO decision.
The market today
Even as the data market heats up to bubbly proportions it is becoming more systemically important to the overall intelligence market. The world’s public data is now incorporated into the models; further improvement will rely on novel methods and novel (private) data.
That private data is sitting completely unaccounted for on company balance sheets and in people’s heads. It has no path towards valuation except in sale, and today that’s mostly selling whole corpus data from failing or failed companies.
Looked at like a financial market the dislocations are clear: there’s no standardized pricing, there’s no standardized unit of value, the spreads are very wide, the transaction costs are very high.
There’s no real way to talk about quality or quantity. And so there’s no standardized way to talk about price (value/volume). Those are all the hallmarks of a wildly immature, underdeveloped market.
If the data market is going to persist - and there are a bunch of reasons to believe it will - it seems inconceivable that we won’t get some dual commoditization and financialization of the inputs for training data, whether that’s raw data or, more likely, more sophisticated and complete byproducts.
The value chain
When you talk about data, you start at the raw data level. On the company side, that’s the full corpus. On the human data side, it’s hours of video or surveys/RLHF tasks from expert networks. Then you add value as you transform it into higher order byproducts, at least in post training.
You turn data into trajectories, the trajectories into tasks, the tasks into full-blown RL environments, etc. At each step, you take this raw, unfiltered substrate and make something more unique and more valuable out of it.
So the worst place to be is the bottom of the totem pole, dealing CSVs with little to no value added beyond origination. The frontier labs’ buying behavior exacerbates this. The big labs only want to buy from a small number of vendors at a time.
-
It’s uneconomic to manage infinite vendor relationships at the scale they’re deploying capital (as much as 9 figures per month for some labs)
-
Transacting directly with many vendors exposes them to increased liability arising from the data provenance
-
It’s impossible to communicate to the market without exposing research priorities and product roadmaps. Working through a limited number of intermediaries helps obfuscate their buying behavior similarly to financial markets where institutions might buy via a dark pool or prime broker offering swaps.
If the spread between a raw corpus and the tasks you build on it is 2 to 5x, the obvious thing to do is not to get raw data and sell it, but to get raw data, build tasks, and sell the tasks. The same is true from tasks to environments, environments to post-training, and post-training to deployment. That’s what’s starting to happen with the biggest data businesses; they are very clearly becoming RL labs and deployment companies for large enterprise (data vendors becoming “data labs”).
The most durably valuable companies in this economy will keep moving downstream, becoming more full stack deployment businesses. The general and evolving trend seems to be companies which help their customers “do AI”. The emerging shape of these businesses seems to be converging on a combination of the data pipeline, deployment and integration, and eventually inference. This is necessary to diversify away from the limited universe of frontier lab customers and to create something with more terminal value - the next gen management and IT services firms.
In parallel, there will be some players that go really deep on a specific part of the value chain around particularly complex industries and OOD abilities/data. But even they will almost certainly hear the call from customers to own more of the “post processing” even if only within a narrow domain.
The bull case
Across the supply chain, the belief that the data market will matter long term basically looks like this:
-
More people will train more models to do more things forever. There will be some more labs. There will be many AI native companies. And many large enterprises will create agents and harnesses that need traces and evals. At least some will post train.
-
Large enterprises and vertical AI companies/neofirms will do this because they don’t want a single point of reliance or failure, view internal information as proprietary and don’t trust labs’ ZDR, and need to bring down costs.
-
Labs will continue buying data to expand the TAM/create new markets for intelligence beyond its core markets today. New domains require new data.
-
There are either limits on synthetic data and/or you need to complement it with real-world data to find out of distribution/novel sources. This will keep the buying robust rather than fully internal to the labs themselves.
Whatever the exact form factor, there will be some long-dated, large, permanent market for data. And it will be macroeconomically important as a key input to the end market for intelligence.
The bear case is obvious. The budgets can’t last. Data is non-rivalrous. Prices get competed down as more sellers wake up, and the labs have to rationalize their spending and show more profitability, so demand shifts left just as supply shifts right. Maybe it all goes synthetic and internally simulated by the labs. Maybe it’s all just harnesses and post-training is a waste of money. Maybe the only businesses with any value are the frontier labs. All of which might be right!
Ultimately the bear case is a repudiation of the current market rather than an indictment of the long term viability of the data market. It is obvious to anyone paying attention that the market regime today is ridiculous, unsophisticated, and not durable - and it does not deserve to last.
The eventual downward price pressure will kick in but the promise of owned intelligence is too tantalizing to ignore and could support a permanent buying pattern sufficient for a large, important data market. You don’t need frontier intelligence for most things, so most of a company’s daily work gets allocated like labor, on ROI, where post-trained models and/or harness + eval win. Getting the most performance from those models means ongoing data needs.
Many sufficiently large software organization will wind up looking like a research org: training agents, buying data and RL environments, etc. in the pursuit of differentiation and cost control. Way more sellers, way more buyers. Price comes down even as volume goes up.
In the bull case, the market for data will expand up to some percentage of the total intelligence market which itself will occupy a rising share of the overall economy. Today, most jobs don’t yet have deep coverage by the models. In other words, we are so early.
via Mercor
Long term, the market that emerges will look basically nothing like it does today. There will be more buyers (labs, neofirms, and enterprises), more standardization, and tighter spreads. Prices for the average corpus will go down precipitously even as overall volumes and dollars go up... at least on the mass market and outside of the rare tokens.
The bifurcated data market
Over time as the market matures and settles into something resembling its terminal state, it will bifurcate into two divergent submarkets.
-
High volume, low value on a unit basis. This will likely approach 100% synthetic and simulated: the Walmart of tokens.
-
Low volume, high value. Here the value comes not from scale but from novelty: the truly out-of-distribution data and the novel signals that emerge from the real world usage, and the edge cases.
That barbell will emerge both within the firm and in the new out-of-distribution data that needs to be acquired. Given the speed at which things are moving with the labs and data business, this split could happen quite quickly (within the next year).
Synthetic data and training pipelines will eat the low value, high volume data and work through the sheer power of scaling laws. Once the tasks and work is within striking distance for the models, incrementally more data can’t/won’t compete against brute force simulation: 1,000,000,000 simulated hours in 1,000,000 runs can smooth out a hell of a lot of kinks.
The real winners in the high volume, synthetic data market will look like extremely talent dense research companies rather than the hustle-y dealmakers in the real world data business. Prices here will collapse in a highly competitive game of measurable gains. It will look and feel like market makers and quant shops.
Contrast that to the out of distribution data, which will continue to be sought after and valued more like VC/PE deals. Hustle, connections, trust, etc. are the law of the land - in addition of course to engineering talent sufficient to extract and add value to the raw substrate.
This coming bifurcation demonstrates why data is not “The New Oil.“
-
First, it’s non-rivalrous. There’s nothing inherent to a CSV that prevents both of us from owning it. You can only enforce data and its byproducts as a rivalrous good via property rights (exclusivity).
-
Second, data and its byproducts are too highly varied to clear in a single market the way oil or compute do. Raw data is a different thing than tasks, and different tasks are different from other tasks or environments. And that’s to say nothing of domains.
So data will become a quasi-commodity. It won’t look exactly the same as other commodity markets, but it will look like more of a commodity market than it does right now: more efficient, more standardized, more liquid, without ever being perfectly efficient, standardized, or liquid.
Building the task factory
Crucially, there will be out-of-distribution data in perpetuity as models refactor the economy and create new operating conditions/environments to study. That is, diffusion itself will keep moving the goalposts. Until a few years ago there was no such thing as an AI-native firm. In the operation of an AI-native neofirm, you are not just relying on data from existing firms but creating an entirely new corpus of interactions that has never existed before, and is therefore definitionally out of distribution.
As the substrate of the economy moves from human hours to tokens and GPU cycles, the machine economy will generate data and outcomes that is not merely better or worse than the human economy’s, but wholly different through the very fact of its being AI produced.
This is the vision for the firm as a Task Factory: as you deploy AI into a business, you are inventing new and novel failure cases for the models, because you’re bringing them up against real-world situations. Refactor the firm to do the transformation, and in so doing you have the telemetry to pull out traces on an ongoing basis. That data will wind up being really valuable, regardless of whether it gets broadly externalized back to the wider supply chain or internalized/reinvested for continued improvement.
Waymo is the historical precedent; it has a huge amount of simulated experience but originates the things it needs to simulate from real-world context. There’s some weird edge case Waymo may only see once/ won’t see often enough in the real world to train on. But Waymo sees it, it can simulate it a billion times and figure out how to solve it. That one real-world instance is novel, valuable, and out of distribution.
We’ll see the same thing in a mid-diffusion economy: agents deployed into real businesses will see an edge case for the first time, and that edge case will be really valuable data for simulation and scaled training.
Pure simulation won’t give you the edge cases. Real world experience won’t give scale.
This is very bullish for AI transformation firms themselves as they are perfect partners to manage that new data capital stock accumulating in the telemetrized, post-transformation firms.
Data native capital
I’ve previously written about the emergence of a data-native capital structure that leverages data as an asset. Today, data is sitting unaccounted for on balance sheets as a $0 intangible. As it becomes more readily bought, sold, and utilized that will change and the market will standardize and financialize.
As units and prices standardize, that filters into book values, valuations, and asset prices. The gap between what data sells for and what it’s carried at is a ton of near-term arbs and, longer term, room for new capital structures. Eventually data is part of the capital structure of businesses alongside credit and equity.
The vehicle that puts it together will be AI rollups with data as a core part of the buying criteria and value creation strategy (CAC is the early leader). Fully financed data - the static corpus and the new potential of the task factory - will be the new junk bond that will spur the new LBO. These data native LBOs will work on the basis of operating leverage (AI transformations) to create new financial leverage.
At the small scale, data native capital will focus on extraction (using the corpus to finance an acquisition). At the large scale, it will focus on origination (using the asset to produce new data through ongoing operation).
With all its tradeoffs and its indeterminate future, data creates substantial option value and new opportunities to be creative and entrepreneurial. It’s a new dimension for capital allocation: data allocation. Making the choice between keeping data internal vs. exposing it externally (and how) is akin to the choice between returning capital to shareholders and letting it compound (and how).
Ultimately the choice to externalize or fully internalize the data a business generates and accumulates will be among the most consequential decisions leaders have to make.
What I want to invest in
I’m fascinated by opportunities to build new businesses and business models around data. If you’re sufficiently data and deployment pilled, there’s a huge number of totally new ideas to go chase.
Ultimately almost all of these companies will become deployment businesses if they succeed. And I love deployment.
The quant side
Simulation and self-play. Businesses that build the environments models run in at scale: whole companies simulated end to end, synthetic customers and counterparties, agents playing against agents. Once the work is within striking distance for the models, this is where the volume comes from: add another million containers and run it again.
Passive monitoring and extraction. Businesses that work through scale by hoovering up real-world data passively. Every business produces data as exhaust, and almost all of it sits idle: find thousands of suppliers, get the permission, structure the data, and get suppliers paid on consumption instead of one corpus sale at a time. Any one supplier’s data is worth little, so it’s a game of coverage, cost, and volume.
Vertical data labs. In sufficiently deep, complex, or out-of-distribution domains, the research and the data origination are each hard enough to justify a whole company. We’ll see players emerge in fields of science (verifiable domains) and non verifiable domains like voice and human preference.
The operations side
Task factories. Real businesses that, in operating, create data that has never existed before. If you do the transformation and the deployment, you are creating a scenario that has not existed before, so every trace that emerges from it is novel by definition.
Incidental producers. Businesses where, in the regular course of business, you either produce or acquire data you can monetize as a secondary income stream, or almost a secondary funding mechanism for your primary business.
Scarce assets. A lot of this winds up saying you need to own IP, access to people, or scarce physical assets as the only things that have any value in a post-AGI world. The dream for Data Native Capital is creating a gem by having a differentiated underwriting lens: an intangible asset sitting on the balance sheet, or a future revenue stream excluded from the DCF that only you can see.
Robotics data. It’s very clear that what’s currently happening in business data will come for robotics data within the next few months and reach a fever pitch over the next few years. The integration ➡️ transformation ➡️ origination playbook will look different in robotics data (it’s more origination than extraction) but the fundamental principle is the same and is very promising.
The financial layer
The token X-ray. Right now the real way you value raw corpus is you get a sample, produce a marketing package around it, send it to two or three people, and ask how much they’ll pay. There’s basically no way to prospectively look at a dataset and say this is how much it’s worth. Which is probably an opportunity. If we can build the token X-ray we can unlock substantial liquidity and volume.
The regular “rules of business” obviously apply to data businesses, obviously. Without some special way to buy (proprietary origination) or sell (value added services) then you’re competing on price and execution. If you have no reason to think there’s gonna be long term margins then that’s just brokerage.
For most businesses, being full-time brokers is a mistake. Lab needs change fast enough that the air goes out of any one thing.
Conclusions
Compute, researchers’ wall clock time, and data are the three primary substrata of the intelligence economy. The market for each will be massive and reinforce the others. Collectively this is the machine economy and data is the third leg of the stool.
While there’s a reasonable case that training data is fully internalized at some point, we’re years away from that. Each new domain and environment that intelligence incorporates/expands to will be further from code and less verifiable - at least on simple measures and short horizons.
So AI diffusion is going to be much slower than the SF groupchat thinks. We’re 25 years into the cloud and it’s not done yet. Migrating to the cloud meant rearchitecting technology systems, which was relatively simple. AI is rearchitecting the economy.
As each org, firm, and industry refactors around intelligence, data (broadly raw data, tasks, evals, traces, etc.) will become a critical link
We’re just really, really early and the race is only just beginning.
As it’s produced, transacted, and consumed in increasing volume data will be priced and valued as an intangible asset or capital good.
For now, the data market is the Wild West: illiquid, inefficient, and full of possibility.
If you can be really aggressive, you can probably hoover up and create a lot of value for yourself while we’re in this highly dislocated market.
The first corpus was the public internet. The next corpus is the whole economy.













