Most of today's foundation models were built in domains where data is abundant. Language models learned from the web. Vision models learned from billions of captioned images. Code models learned from every public repository ever pushed. CAD is different.
Parametric CAD data is scarce, proprietary, and expensive to label. And yet nearly every physical product around us starts its life in CAD. If we want AI to transform physical engineering the way it transformed software, we have to find a way to learn without the exabytes of scraped data and large-scale human-labeling efforts that powered everything else.
So we asked a simple question: can a model learn from the structure that is already embedded inside a CAD file? Our answer is Masked Topology Modeling (MTM), described in our new paper, Masked Topology Modeling for Self-Supervised Learning on Parametric CAD.
CAD files are already labeled
A boundary representation (B-rep) solid is more than a bag of surfaces. It is a graph: every face is a node, and every pair of faces that share an edge is connected. Crucially, each of those edges carries rich information that the geometric kernel computes deterministically, with no human in the loop.
- Convexity — do the two faces meet at a convex edge, a concave edge, a smooth (tangent) transition, or a knife edge?
- Curve type — is the shared edge a line, a circle, a B-spline, an ellipse, or something else?
These labels encode real engineering semantics. A concave circular edge between a planar face and a cylindrical face is, more often than not, a drilled hole. A ring of convex line edges is the top of a boss. The kernel hands us millions of these annotations for free on every model we can get our hands on.
Masked Topology Modeling
MTM is a pretraining objective in the spirit of masked language modeling, but on topology rather than tokens. We take the face-adjacency graph, hide a large fraction of its edges (we mask 70%), encode the faces with a graph-attention and transformer stack, and ask the model to predict the convexity class and curve type of every hidden edge from the face features alone.
To answer correctly, the model has to understand how surfaces relate in 3D: which faces touch, how they meet, and what kind of feature they form together. That is exactly the kind of spatial reasoning downstream tasks need. We combine MTM with a contrastive objective and masked feature reconstruction, and pretrain a 27.9M-parameter encoder on roughly one million shapes from the ABC dataset plus a procedurally generated sketch-and-extrude corpus.
Results
We evaluate on four benchmarks that test whether a model can recognize the actual features of a part: per-face segmentation on Fusion 360 Gallery and MFInstSeg, dense feature annotation on CADSynth, and shape classification on SolidLetters.
With full supervision, MTM sets a new state of the art on Fusion 360 Gallery segmentation, outperforming both supervised BRepNet and the self-supervised BRep-BERT.
But the headline result is what happens when labels run out. This is the regime that actually matters for CAD, where a labeled dataset might be a few hundred parts annotated by a single engineer.
The biggest gains show up exactly where labels are scarcest. With a tenth of a percent of the MFInstSeg labels, MTM is already at 87 mIoU, where the strongest supervised baseline is still below 60. At ten shots on Fusion 360 Gallery, the gap is more than three to one.
Why this matters
To me, the bigger implication is that scaling AI for physical engineering may not require copying the playbook from language or vision. The structure we need is already in the data. The geometry kernel is, in effect, an oracle that labels every edge of every model ever built, and self-supervision lets us turn that into a learning signal.
No human labels
Every MTM target is computed by the kernel. Pretraining scales with the number of CAD files we can collect, not the number of engineers we can hire.
Labels go further
A pretrained encoder makes a few hundred annotated parts worth what tens of thousands were worth before.
Geometry, not pixels
Operating on B-rep topology preserves the engineering intent that meshes and renders throw away.
MTM is one piece of the foundation we are building at the intersection of AI, geometry, and physical engineering. Read the full paper, and if you are working on CAD data and want to compare notes, reach out at [email protected].
