NeurIPS 2026 Submission

C2CAD-Bench: A Large-Scale Benchmark for 3D Spatial Reasoning

We remove the code layer entirely. Models read natural-language engineering specifications and return deterministic JSON geometry, scored against parametric ground truth — no execution, no rendering, no shortcuts.

Anonymous Authors 27 frontier LLMs 10 providers

Paper Code Explore 3D

Test families

Test cases

Frontier LLMs

805

Primitives in one case

Benchmark snapshot

Aggregate across 27 frontier models

Live data

75%

Mean score

Coverage 85%

Geometry 73%

Semantic 76%

Top models

01 Claude Opus 5.5 95

02 Claude Sonnet 5.5 94

03 Step 5 Preview 85

Coverage–geometry gap 12 pts

Abstract

Spatial understanding, measured without code.

Existing benchmarks blend programming proficiency with geometric reasoning. C2CAD-Bench separates the two so that a syntactically correct script that places every part at the wrong coordinate no longer passes.

AI-assisted CAD design relies on a prompt-to-code-to-CAD pipeline, where an LLM generates code for a CAD engine from a natural-language description. Existing benchmarks combine programming proficiency with spatial understanding; reaching a syntactically correct script that places every part at the wrong coordinate passes all execution-based checks. Moreover, image-based evaluation cannot inspect internal geometry such as lattice connectivity or clearance fits.

We introduce C2CAD-BENCH, a benchmark that isolates 3D spatial reasoning from code-generation proficiency by removing the code layer entirely. Models receive a natural-language engineering specification and must produce a JSON array of geometric primitives, scored deterministically against parametric ground truth without code execution or rendering. Prompts follow a zero-scaffolding methodology that supplies only physical intent, requiring models to derive every coordinate from first principles.

The benchmark comprises 25 test families across four phases of cognitive demand (75 test cases, up to 805 shapes), evaluated with a gated three-axis scoring pipeline — Coverage, Geometry, and Semantic. Evaluation of 13 frontier LLMs from five providers reveals a persistent coverage–geometry gap of roughly 19 points, systematic primitive type confusion, and that constraint complexity rather than shape count drives difficulty.

Why C2CAD-Bench

Three changes that make reasoning measurable.

A benchmark is only as good as what it isolates. We rebuilt the evaluation target around the part of the problem that actually matters: spatial intent.

No code layer

Models emit a JSON array of primitives instead of a program. Correctness no longer depends on API fluency, import paths or execution environments.

Deterministic scoring

A gated three-axis pipeline compares coverage, geometry and semantics against parametric ground truth. Every score can be recomputed from the same inputs.

Cognitive depth

Four phases move from single primitives to functional assemblies, exposing where constraint complexity — not shape count — breaks the reasoning chain.

Dimension	Execution-based	Image-based	C2CAD-Bench
Signal evaluated	Program validity + output	Rendered appearance	Parametric geometry
Code required	Yes	No	No
Internal geometry	Limited	Not visible	Fully inspected
Scoring	Pass / fail or rendered similarity	Image metrics	Gated, deterministic, three-axis
Reproducibility	Environment dependent	Render dependent	Exact

Benchmark structure

25 families across four phases of cognitive demand.

Each family contains three difficulty levels. Phase 1 focuses on primitives and repetition; Phase 4 pushes toward functional assemblies and extreme-scale patterns.

Phase 1 18 cases

Foundations

Primitive placement, repetition, radial patterns and basic constraints.

Spiral StaircaseCannonball PyramidVoxel Grid Domino RingDNA HelixFlanged Pipe Joint

Phase 2 18 cases

Structures

Tension, bridging, lattices and orbiting component relationships.

Suspension BridgePlanetary ArrayCross-Braced Truss Fractal Y-TreeBCC LatticeBall Bearing Assembly

Phase 3 18 cases

Mechanisms

Multi-part assemblies, concentricity, clearances and functional grouping.

Furniture AssemblyPipe ManifoldAxle Bearing Armillary SphereClock Tower MechanismGantry Crane Assembly

Phase 4 21 cases

Extreme scale

Algorithmic growth, organic geometry and cases with up to 805 primitives.

Phyllotaxis DiscCompound EyeDiatom Frustule Honeycomb LatticeVertebral ColumnCochlear Spiral Radiolarian Skeleton

Spiral StaircaseL1–L3 Cannonball PyramidL1–L3 Voxel GridL1–L3 Domino RingL1–L3 DNA HelixL1–L3 Flanged Pipe JointL1–L3 Suspension BridgeL1–L3 Planetary ArrayL1–L3 Cross-Braced TrussL1–L3 Fractal Y-TreeL1–L3 BCC LatticeL1–L3 Ball Bearing AssemblyL1–L3 Furniture AssemblyL1–L3 Pipe ManifoldL1–L3 Axle BearingL1–L3 Armillary SphereL1–L3 Clock Tower MechanismL1–L3 Gantry Crane AssemblyL1–L3 Phyllotaxis DiscL1–L3 Compound EyeL1–L3 Diatom FrustuleL1–L3 Honeycomb LatticeL1–L3 Vertebral ColumnL1–L3 Cochlear SpiralL1–L3 Radiolarian SkeletonL1–L3

Interactive showcase

Ground truth vs. the top four models.

Eight landmark cases with high performance variance. Every viewport is live — drag to orbit.

Open full visualizer

Loading live benchmark geometry…

Ground truth models are rendered from the parametric reference; model outputs are reconstructed from returned JSON primitives.

Global evaluation

Leaderboard across 27 frontier models.

Scores are aggregated over all 75 cases. Expand any family to inspect the Coverage, Geometry and Semantic breakdown.

Full scoreboard

Loading evaluation data…

Strong (> 80) Partial (40–80) Weak (< 40)

Analysis

What the numbers reveal.

Across 13 models, coverage is consistently easier than geometry. The gap widens with constraint complexity, and primitive confusion remains the dominant failure mode.

Coverage vs. geometry

19 pts

Models find the parts, not the places.

Mean coverage is 84% while mean geometry is 65%. Producing the right primitive types is markedly easier than placing them correctly.

Constraint complexity

−18 pts

Assembly phases are the inflection point.

Phase 1 and 2 averages sit near 80%, while Phase 3 and 4 fall to roughly 61–64% as clearances and dependency chains accumulate.

Failure taxonomy

Primitive type confusion leads errors.

Models systematically substitute one primitive for another, then build a plausible but geometrically invalid assembly around the mistake.

Evaluation protocol

One score, three gated axes.

Scoring is deterministic and inspection-only. Each axis isolates a different failure mode, and geometric credit is gated by primitive matching.

AXIS 01

Coverage

Did the model produce every primitive implied by the specification?

Cov = matched / |GT|

AXIS 02

Geometry

Are centers, dimensions, axes and orientations within tolerance?

Geom = 1 − normalized error

AXIS 03

Semantic

Does each primitive carry the right role and participate in the right relationships?

Sem = weighted role match

Gated scoring. Geometry is only credited for correctly matched primitives; coverage cannot be redeemed by plausible-looking extra geometry. Final scores are the weighted aggregate of the three axes.

Citation

Use C2CAD-Bench in your research.

If this benchmark helps your work, please cite the paper.

View on GitHub

BibTeX

@article{c2cad2026,
  title   = {C2CAD-Bench: A Large-Scale Benchmark for 3D Spatial Reasoning in CAD},
  author  = {Anonymous Authors},
  journal = {arXiv preprint arXiv:26xx.xxxxx},
  year    = {2026}
}

Inspect every primitive yourself.

The 3D visualizer lets you compare ground truth against any model, switch difficulty levels, and inspect the exact prompt behind each case.

Open 3D Visualizer Scoreboard