Confident and Wrong: Why Protein Structure Models and the Frontier Labs Can’t Code Protein Function
文章目录
Billions of dollars are flowing to protein discovery, from seed-stage startups to Isomorphic Labs, DeepMind’s drug-discovery spinout, to Anthropic, which bought a biotech and now runs its own wet lab. But no amount of capital or compute can conjure up data that was never collected.
Our new research shows the misguided nature of the industry’s leading efforts: These models have gotten very good at telling us what proteins look like, but that is not the same as knowing what they do.
Today, the Aether team is publishing our first paper: "Confident and Wrong: Structure Metrics Fail To Predict Metal Binding Across Protein Design Strategies."
The short version: the designs that looked most confident produced the fewest working proteins. The design set with the lowest confidence scores produced the most working proteins, and the only designs that beat what nature already offers.
To test this, we picked a hard, real target: getting a protein to tell terbium and dysprosium apart. They sit next to each other in the rare earth series with nearly identical chemistry, and both go into the high-performance magnets inside motors, generators and actuators. Separating them today takes long solvent extraction trains that are capital-heavy, chemically toxic, and concentrated in very few places.
We started from lanmodulin, a natural protein that binds rare earths tightly and selectively. Then we ran three design methods under the same conditions: same starting protein, same regions allowed to change, same expression, same assay.
Two methods picked candidates based on how confident a structure model was in the predicted shape (RFdiffusion with confidence filtering, and DISCO, which co-generates sequence and structure around the metal). The third, ours, used our Protein Function Model, trained directly on measured binding, to rank candidates.
The results.
Designs that showed binding:
-
Aether: 83%
-
DISCO: 61%
-
Structure-confidence: 16%
Median terbium/dysprosium selectivity: 1.70 for ours, 1.38 for DISCO, 1.42 for structure-confidence, against 1.26 for natural lanmodulin variants across 149 measured proteins. Only our designs were distinguishable from nature (p = 5.6e-16); neither structure-guided method consistently was.
1.70 sounds modest. It isn't. Separation runs many stages in series, so small per-stage gains multiply.
Now the uncomfortable part.
We tested every standard structural confidence metric: pTM, ligand ipTM, complex ipLDDT, iPDE, Boltz-2 confidence, and the Boltz-2 affinity heads. Across hundreds of measured proteins, they did not just correlate weakly, they flipped sign between design sets (rank correlations against measured binding from about -0.45 to +0.38). There is no prospective rule you can extract from that. You cannot know which direction to filter until after you have measured.
We even trained a model on the structural metrics themselves, to give them their best shot. It didn't generalize. The set with the highest confidence scores had the lowest rate of working proteins. It expressed well. It scored highest on every confidence metric. And 84% of it didn't bind. A team running that pipeline without a measurement loop would have reported success.
Our Protein Function Model, by contrast, gave the best classification performance in every design set (ROC AUC of 0.92, 0.79 and 0.88), on proteins it had never seen.
Two limitations, both stated in the paper: we ran all three design arms in-house, and the two comparison arms were sampled one sequence per backbone.
Why this matters.
Modern protein design runs on a shared assumption: predict the shape, score your confidence in it, keep what scores well. It's everywhere, because it's the only filter most groups have. As a result, the field has quietly been treating a confidence number about geometry as proof that a protein will do its job.
On this target, that assumption fails. The filter kept the wrong proteins. You can generate ten thousand candidates and still have no way to rank them, because the signal was never in the structure.
To be fair, this isn't a failure of the models. The failure is in using a confidence score as a stand-in for function. But what’s held back the industry from going further is a data gap.
Structure models learned from a huge public library of solved structures. There is no equivalent public library of controlled functional measurements. Whoever builds it gets a capability that can't be downloaded.
That's where we’ve focused the better part of our seven years at Aether. The model that won here trained on roughly 2,500 proteins measured in-house: sequence, metal, buffer, and assay conditions in, measured binding out. It learns what a protein does under real conditions, not what it looks like.
Our view is simple: chemistry should be programmable. Specify what a molecule needs to do, get a design that does it, make it at scale. Proteins are the most programmable chemistry we have. The blocker was never generating candidates, but knowing which ones will work before you spend six months finding out. A protein that prefers terbium over dysprosium is the front end of a very different kind of separation plant.
Confidence in a predicted shape is not evidence of function. It's time we stopped treating it like it is. Because ultimately the goal is not to design proteins that look right, but that work in the real, messy world.
Paper here: https://arxiv.org/abs/2611.17391
Huge thanks to everyone on the Aether team behind every one of these experiments.



