Every frontier model release comes with impressive coding benchmarks. GPT-5 is no exception — and parametric CAD generation is, on its face, a coding task: the model emits a sequence of sketch and feature operations that a kernel replays into a solid.
So we asked the obvious question: if frontier models can write code this well, can they build objects?
The benchmark
We evaluated GPT-5 on a held-out slice of our parametric CAD dataset: natural-language and image prompts paired with ground-truth parametric models, spanning brackets, housings, gears, and multi-body assemblies.
- Text-to-CAD — generate a parametric model from a written spec.
- Image-to-CAD — reconstruct an editable model from renders of the target part.
- Edit tasks — modify an existing feature tree while preserving constraints.
Syntax is solved
Let's give credit where due: GPT-5's output is essentially always well-formed. Scripts parse, APIs are called correctly, and the code executes without error in over 97% of attempts. On syntax, frontier models are done.
Geometry is not
Execution success is not geometric success. Measured by Intersection-over-Union against ground truth, the picture changes sharply.
Failure modes
The errors are not random. They cluster into patterns any mechanical engineer would spot instantly: extrusions in the wrong direction, holes placed by symmetry the prompt never implied, fillets applied to phantom edges, and dimensions that are plausible tokens rather than measured quantities.
The model has learned what CAD code looks like, not what space is. It interpolates over programs, not over geometry.
Why this happens
Internet-scale pre-training contains an enormous amount of code and text about 3D, and almost none of the dense, supervised pairing between operations and resulting geometry that spatial understanding requires. This mirrors what we found in our LoRA fine-tuning experiments: 3D reasoning is far enough outside the pre-training distribution that it has to be learned, not adapted in.
Conclusion
General-purpose frontier models are extraordinary programmers and poor geometers. Closing that gap is not a prompting problem — it requires training on parametric CAD at scale, with geometric supervision. That is exactly the model we are building.
