--- license: apache-2.0 tags: - vision-language - image-to-cad - cad - cadquery - qwen2-vl - 3d - point-cloud - multimodal --- # CADReasoner-CM (cross-modality variant) CADReasoner-CM is a variant of [CADReasoner](https://huggingface.co/kulibinai/cadreasoner), a vision–language model for **iterative CAD reverse engineering**. It generates a **runnable CadQuery program** and refines it over several iterations using geometric feedback from the discrepancy between the target shape and the current prediction. This checkpoint conditions on **both the point cloud and multi-view renders** of the target shape. Combining the two modalities improves geometric alignment and recovers fine details that either modality alone tends to miss. **Accepted to CVPR 2026 Findings Track** **Paper:** https://arxiv.org/abs/2603.29847 **Code:** https://github.com/zhemdi/CADReasoner (see the `pc_cm/` directory) **HF paper page:** https://huggingface.co/papers/2603.29847 ## Variants | Model | Input modality | Flags | |---|---|---| | [`kulibinai/cadreasoner`](https://huggingface.co/kulibinai/cadreasoner) | multi-view renders | — (root scripts) | | [`kulibinai/cadreasoner-pc`](https://huggingface.co/kulibinai/cadreasoner-pc) | point cloud | `--use_pc true --use_img false` | | [`kulibinai/cadreasoner-cm`](https://huggingface.co/kulibinai/cadreasoner-cm) | point cloud + multi-view renders | `--use_pc true --use_img true` | ## Architecture This checkpoint uses the `Cadrille` architecture: a Qwen2-VL-2B backbone plus a Fourier point-cloud encoder. The point features are projected to the hidden size and **scattered into a run of pad tokens prepended to the prompt**, so the model cannot be loaded with a bare `Qwen2VLForConditionalGeneration.from_pretrained` — `config.json` declares `architectures: ["Cadrille"]`, and the class lives in `pc_cm/cadrille.py` in the repository. The point cloud is built from the **discrepancy** between the target mesh and the current prediction, not from the target alone: both surfaces are sampled, points whose distance to the opposite shape exceeds a percentile threshold are kept, and farthest point sampling reduces them. Each direction contributes `n_points` points — `n_points` from GT→pred and `n_points` from pred→GT — so the prompt reserves `2 * n_points` pad tokens and each point carries 6 features (position + displacement vector). On the first iteration, where no prediction exists yet, the bounding-box centre is used in place of the predicted mesh. `--n_points` must match the value used in training: **128**. ## Setup ```bash git clone https://github.com/zhemdi/CADReasoner.git cd CADReasoner ``` Build the environment from the provided `Dockerfile`, then add the two packages the `pc_cm/` scripts need on top of it: ```bash pip install opencv-python rtree ``` `opencv-python` is used by the image augmentations, and `rtree` backs trimesh's closest-point queries — without it the code silently falls back to a KD-tree over mesh vertices, which is less accurate on coarse meshes. The scripts load the model with `attn_implementation="flash_attention_2"` and `torch.bfloat16`, so an Ampere-or-newer GPU with `flash-attn` installed is required. Inference shards samples across all visible GPUs, one process per GPU, and needs at least one. ## Inference ```bash python3 pc_cm/test.py \ --dataset \ --checkpoint kulibinai/cadreasoner-cm \ --use_pc true --use_img true \ --n_points 128 \ --n_iters 3 \ --n_samples 4 \ --outdir preds_cm ``` `` is either a local directory of `.stl` files or a Hugging Face dataset repo id: * `maksimko123/deepcad_test_mesh` * `maksimko123/fusion360_test_mesh` * `kulibinai/mcb_test` * `kulibinai/deepcad_test_scan` * `kulibinai/fusion360_test_scan` * `kulibinai/mcb_test_scan` Each iteration writes a CadQuery program and its compiled mesh per candidate: ```text preds_cm////.py preds_cm////.stl ``` Files are renamed to their chamfer distance so the next iteration can pick the best candidates to refine. `--n_samples` controls how many candidates are sampled per shape (the first is greedy, the rest use temperature 1.2). ## Evaluation ```bash python3 pc_cm/evaluate.py \ --dataset \ --pred_dir preds_cm/ ``` For each refinement iteration this reports median chamfer distance (scaled by 1000), mean IoU, and the invalidity ratio — the fraction of shapes with no valid prediction. Each shape is scored with the best candidate seen up to that iteration, so the numbers are cumulative across iterations. ## Training The curriculum runs over dataset groups `0, 1, 2` (see `data/README.md` for preparing the split). Group 0 starts from `Qwen/Qwen2-VL-2B-Instruct`; each later group starts from the previous group's final checkpoint. ```bash torchrun --nproc_per_node pc_cm/train.py \ --dataset_dir \ --use_pc true --use_img true \ --n_points 128 ``` Checkpoints and logs are written to `runs//_/`, with per-group weights under `model//final`. Pass `--run_dir` and `--groups` to resume an interrupted run, and `--skip_generate_code` / `--skip_generate_meshes` to reuse refinement samples already on disk. ## Loading the model directly If you are integrating the model rather than using the scripts: ```python import torch from transformers import AutoProcessor from cadrille import Cadrille # from pc_cm/ processor = AutoProcessor.from_pretrained( "kulibinai/cadreasoner-cm", resized_width=14 * 17 * 2, resized_height=14 * 17 * 4, padding_side="left", use_fast=True, ) model = Cadrille.from_pretrained( "kulibinai/cadreasoner-cm", torch_dtype=torch.bfloat16, attn_implementation="flash_attention_2", ).cuda().eval() ``` `forward` and `generate` additionally require `point_clouds` (a `(batch, 2 * n_points, 6)` tensor), `is_pc` and `is_img` — they are not optional, and the pad-token prefix must already be present in the prompt. See `generate_predictions_process` in `pc_cm/test.py` for a complete, working example. ## Limitations The image-conditioned [`kulibinai/cadreasoner`](https://huggingface.co/kulibinai/cadreasoner) gives the best results in our implementation; the geometry-conditioned variants are released for reproducibility and for settings where rendered views are unavailable. The point-cloud modality also proved unstable under RL fine-tuning, so it is not being developed further at present. ## Citation ```bibtex @InProceedings{Kabisov_2026_CVPR, author = {Kabisov, Soslan and Kirichuk, Vsevolod and Volkov, Andrey and Barannikov, Marina and Savrasov, Gennadiy and Konushin, Anton and Kuznetsov, Andrey and Zhemchuzhnikov, Dmitrii}, title = {CADReasoner: Iterative Program Editing for CAD Reverse Engineering}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings}, month = {June}, year = {2026}, pages = {6143-6153} } ```