Robotics Data: Bet Hard Or Get Out
文章目录
部分视频或图片暂未收录，已在正文保留提示；需要时可查看原文。查看原文 ↗
On the survivability of Robotics Data Collection companies and what comes after the wave.
Since posting my "Robotics Data is a Broken Business" essay, my DMs and calendar have been filled with conversations with founders, researchers, investors, and buyers across the robotics ecosystem.
It's been awesome.
I’ve spoken with so many brilliant people, many of whom have taught me far more than I’ve given back, and I’m genuinely grateful to everyone who’s taken the time to share how they’re thinking about the field.
Across those conversations, though, I see a pattern emerging. And it is shaping my thesis around where robotics data companies can survive long-term, and where I’ll personally build next.
It has two shapes: A pyramid and an hourglass.
Three Ways Forward
When we first started @Eidon_AI in February of 2024, we were going after multimodal data. At a time when multimodality was not guaranteed in most frontier models, we bet that long-tail videos and images would be extremely valuable.
We built an app that gave users data-collection tasks for images and videos that were outside the normal distribution of the training corpus at the time. Hands, POV, close-ups, different home configurations, different countries, etc.
We eventually collected over 1 TB of over 90,000 data points from over 12,000 individual collectors. Truly long-tail data. But when it came time to commercialize it, we ran into an issue.
Beyond the systemic market challenges I mentioned in the previous essay (a small buyer network, short useful lifespan of data, etc.), we realized that our dataset sat in an awkward middle.
It wasn't bespoke enough to stand on uniqueness alone, and it wasn't large enough to stand on size. It was rare and net-new data (unlike the scraping approach many companies back then were following), which made it hard and slow to collect, but it was neither sufficiently large nor sufficiently unique.
I see a very similar limbo emerging with egocentric robotics data.
Scale
500,000 hours. That is the approximate number I heard from a few people this week for the total amount of data a lab consumes in a month.
500,000 hours per month. If you assume the entire data production industry flows up to a handful of consumers buying the same corpus of non-exclusive data, then this is a solid proxy number for the aggregate of what the entire industry produces.
Depending on how you look at it, that number is puny or ginormous.
From the perspective of any individual player, the number is huge. No data collector in the market reliably produces anywhere close to that. Most sit two, if not three, orders of magnitude below that number. And even the biggest data aggregator is probably responsible for, at most,20% of the market total.
But from the perspective of the entire industry's output, the number is borderline laughable. For comparison, TikTok is estimated to produce 11.4M hours per month. YouTube is 21.6M per month.
Robotics ego data monthly vs. TikTok vs. YouTube
Great news for a robotics data startup, right? There is still a huge gap to close to reach true "internet scale". It would be, were it not for existing giants with tens of millions of active users who are seriously considering entering the space - some of whom I've spoken to.
For them, doubling the industry's output is a single feature launch away. And I don't think I need to say anything about how hard it is to compete with any of these giants.
The scale play works. If you think your data startup can reach even just hundreds of thousands of hours per month, you may have a shot.
Specificity
One of the main takeaways from my last essay was to do hard things. Hard things create moats. The tradeoff is that they often don't scale easily.
But they may not need to.
Not all robotics data is created equal. An hour of egocentric video is very different from an hour that also captures precise hand motion, contact, pressure, force, slip, and deformation.
While the scale bet is about maximizing hours, specificity is about maximizing information per hour. Collect less data, but make each hour contain physical information that cannot be derived from video alone.
This is especially interesting because most of today's models already carry enormous visual and semantic knowledge. Much of which - as we have seen with the Astra demos - translates into some level of visuospatial understanding out of the box (strongly recommend @Georgiachal's post on this: https://x.com/GeorgiaChal/status/2101335513558929868).
There is research pointing in this direction too.
T-Rex, a recent project from researchers across Berkeley, NVIDIA, and Stanford, is the perfect example. Its training corpus combined ~23k hours of human demonstrations with just 100 hours of much richer tactile-synchronized robot data.
The human demonstrations give the model broad visual and manipulation understanding, while the smaller tactile dataset grounds that knowledge in contact and force.
This doesn't mean that rich tactile data eliminates the need for scale. But it shows that different modalities contribute disproportionately different value at very different scales.
@OpenGraph_Labs is a company making this bet today. They are building tactile gloves and synchronized multimodal collection systems around them.
This data is much harder to collect than pure egocentric data. Much harder also than the data we collected at @Eidon_AI with IMUs and IK. The hardware is harder to manufacture, calibrate, deploy, and maintain. But that's also the point. As I've said many times, hard things create moats.
The problem is that specificity goes both ways.
Data becomes more valuable the closer it matches the specific embodiment the model trained on it will control. The sensor stack, degrees of freedom, or manipulation tasks. And this creates a strong incentive for robotics companies to just own the collection stack themselves.
Full-stack robotics companies like @mimicrobotics, @sundayrobotics, and @gs_ai_ all have announced data collection gloves built specifically for their embodiment.
Mimic Robotics' Umimic:
[未转换内容，参见原始 JSON]
Sunday Robotics' Skill Capture Glove via @tonyzzhao:
Genesis AI's 1:1 Data Glove:
If data collection becomes specific to the embodiment, the business model for the companies doing it as a third party evaporates.
Synthetic
The last path forward I see is less a new way to collect data and more an alternative to collection itself. Synthetic data tries to turn the data bottleneck into a function of GPU hours instead of logistics.
Real-world data becomes the seed of a larger, synthetic corpus.
I spoke to companies actively doing this. They buy relatively small egocentric datasets and build their own generation pipelines on top of them, and then train on the entire corpus.
Admittedly, I have not heard of any data company successfully selling a synthetically expanded dataset.
Synthetic trajectories generated in Isaac Lab by the GR00T team.
NVIDIA's GR00T is a good example of what this looks like end-to-end. They started from a small number of human demonstrations and built a pipeline to transform and replay those behaviors across new object positions and tasks. They generated ~6,500 human-hours equivalent of data and reported that the mixed corpus improved GR00T N1 performance by 40% versus real data alone.
Whether the improvements from synthetic data reliably translate from sim to lab is up for debate. But beyond that, there is an obvious problem for any founder considering synthetic data as a business bet.
NVIDIA Isaac GR00T-Mimic blueprint
If your buyers are sophisticated enough to build frontier robotics models, it is likely they will also be sophisticated enough to build a synthetic data pipeline around their own models and fleet.
In many ways, this is an even easier thing to bring in-house than physical collection. The logistics overhead is minimal, and the talent already exists at these labs. NVIDIA is a perfect example.
The robotics data bottleneck will not be solved anytime soon. But the window of opportunity to start, scale, and exit a brand new robotics data company is closing rapidly if it has not already closed.
For those still in the space, I see three bets with better odds: Scale, Specificity, or Synthetic. For everyone else in the awkward middle, I don't see a path forward.
And for those now thinking about getting in, I'd get out.
The Hourglass of Attention
In all the conversations I've had over the last two weeks, the question I've been asked the most is, by far:
If you were to start over in data today, what would you do?
To which I have consistently answered:
I wouldn't. I would go do something else in the space.
Robotics attention has a weird hourglass shape: two big ends and a thin middle. At one end you have the flashy demos, the robot olympics, the robot fights, backflips, and dancing robots. That is the industry's business card for outsiders.
At the other end, you have the entry point for the swath of people venturing into robotics from other industries: Data collection.
The middle - which has many of the hardest unsolved problems in the field - receives surprisingly little attention. The middle is everything needed to turn field data into a deployed robot. The list is big.
Training and architectures: VLAs, World Models, and different ways of learning from physical, video, and synthetic data.
Inference and control: Making increasingly capable models fast and cheap enough to run close to a deployment site, or eventually on the final embodiment. And figuring out how larger generalist models interact with smaller, faster control policies.
Generalization: Getting policies to work across tasks, object configurations, environments, lighting conditions, and even embodiments. And reducing the drop from simulation to lab and from the lab to deployment.
Evals: Building standards and benchmarks across sim and real-world evaluation that actually tell us whether a system is improving, how it compares with alternatives, and whether those improvements survive outside controlled settings.
And much more: Deployment infrastructure, hardware, supply chains, actuators, sensors, safety, reliability.
None of this even touches the much larger world of non-generalist robotics: autonomous vehicles, drones, industrial arms, delivery robots, and so on. These systems are here, now, at significant scale. And entire sub-industries have emerged around the infrastructure required to operate, maintain, and improve them.
Generalist robotics will likely need its own version of that ecosystem.
But it is very much because of this hourglass attention shape that many newcomers to the space misunderstand how far along generalist robotics truly is.
Even if you set aside the economics of deploying general-purpose robots at scale, generalization is very much not solved. Published task completion rates often collapse by several percentage points when the object configuration, environment, or lighting conditions change slightly.
@JitendraMalikCV's 5-year-old test is a good example.
Parting Thoughts
There are still great companies to be built in the robotics data layer. Just because we failed, it does not mean that other people can't succeed. And the three bets I described above are what I see as a viable future for industry players.
But I also think that we - from engineers and researchers to founders and investors - have collectively over-indexed on data as *the thing *holding back the field.
Data is a bottleneck. But it is not the only one.
There is so much in that middle to figure out, and we need more brilliant minds entering that space, solving hard problems and building great companies along the way.
That's where I'm looking for my way back into the industry.
I'm particularly interested in new training techniques and architectures, and in the infrastructure around turning physical data into reliable behavior.
I don't know exactly what that looks like yet, but I know I want to be in that middle.
