model card: single-pass build, retract double-quant caveat
Browse files
README.md
CHANGED
|
@@ -108,32 +108,42 @@ offloaded to CPU after encoding — it runs once per prompt, not once per sampli
|
|
| 108 |
|
| 109 |
## How `pruned_nvfp4` was built
|
| 110 |
|
| 111 |
-
|
| 112 |
-
|
|
|
|
| 113 |
|
| 114 |
```python
|
| 115 |
-
|
| 116 |
-
|
| 117 |
```
|
| 118 |
|
| 119 |
-
|
| 120 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 121 |
|
| 122 |
```python
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
is_weight=True, convrot=True, convrot_groupsize=256)
|
| 126 |
-
deq = ckt.TensorWiseINT8Layout.dequantize(qdata, params) # -> bf16
|
| 127 |
-
nq, nparams = ckt.TensorCoreNVFP4Layout.quantize(deq.contiguous()) # -> NVFP4
|
| 128 |
-
tensors = ckt.TensorCoreNVFP4Layout.state_dict_tensors(nq, nparams)
|
| 129 |
```
|
| 130 |
|
| 131 |
-
|
| 132 |
-
`
|
| 133 |
-
|
| 134 |
-
|
| 135 |
-
|
| 136 |
-
Full script: `pruned_to_nvfp4.py` in this repo. 200 layers, ~6 seconds on one GPU.
|
| 137 |
|
| 138 |
## Prompting: H3 wants a structured IR, not prose
|
| 139 |
|
|
@@ -217,6 +227,10 @@ frame `length` must satisfy 17n+5.
|
|
| 217 |
VRAM (~20 GB vs ~12 GB) and can spend ~14% more time per step, Comfy-Org's
|
| 218 |
`pruned_int8_convrot` may be the better choice. This repo's value is smallest-footprint
|
| 219 |
and fastest, not highest-fidelity.
|
|
|
|
|
|
|
|
|
|
|
|
|
| 220 |
- That comparison is **not controlled**: the two runs also differed in text encoder
|
| 221 |
(nvfp4_awq vs int8_convrot), it was n=1, and matched seeds do not produce matched
|
| 222 |
trajectories across different quantization. Treat it as a lead, not a result.
|
|
@@ -226,9 +240,9 @@ frame `length` must satisfy 17n+5.
|
|
| 226 |
- 15s (362 frames) generates fine; see the VRAM note below.
|
| 227 |
- The **`fl2va` file is unbenchmarked** — it was produced by the same script over the same
|
| 228 |
layer set as `ref2va`, but has not been run end-to-end.
|
| 229 |
-
-
|
| 230 |
-
|
| 231 |
-
|
| 232 |
|
| 233 |
Failure cases are welcome in the discussions tab — concrete artifacts beat aggregate scores.
|
| 234 |
|
|
|
|
| 108 |
|
| 109 |
## How `pruned_nvfp4` was built
|
| 110 |
|
| 111 |
+
**Single pass from `pruned_bf16`.** Comfy-Org's `minimax_h3_{ref2va,fl2va}_pruned_bf16`
|
| 112 |
+
(40.2 GB) is full-precision weights with AdaLN already refactored into the timestep table, so
|
| 113 |
+
the 200 attn/mlp layers are quantized once, straight from bf16:
|
| 114 |
|
| 115 |
```python
|
| 116 |
+
nq, nparams = ckt.TensorCoreNVFP4Layout.quantize(w.contiguous()) # bf16 -> NVFP4
|
| 117 |
+
tensors = ckt.TensorCoreNVFP4Layout.state_dict_tensors(nq, nparams)
|
| 118 |
```
|
| 119 |
|
| 120 |
+
The layer set is `blocks.<N>.{attn.qkv_proj, attn.out_proj, mlp.fc1, mlp.fc2}.weight` — 50
|
| 121 |
+
blocks x 4 = 200, verified set-identical to the layers Comfy-Org quantize in their
|
| 122 |
+
`pruned_int8_convrot` release. Everything else (AdaLN, norms, embedders, token_refiner, all
|
| 123 |
+
biases) is copied verbatim at source precision: 332 tensors, 1132 in the output.
|
| 124 |
+
|
| 125 |
+
Per-layer config lives in a `comfy_quant` uint8 tensor holding JSON, written as
|
| 126 |
+
`{"format": "nvfp4"}`; there is no file-level `__metadata__`.
|
| 127 |
+
|
| 128 |
+
Full script: `bf16_to_nvfp4.py` in this repo. 200 layers in ~5 seconds on one GPU.
|
| 129 |
+
|
| 130 |
+
<details>
|
| 131 |
+
<summary>Earlier builds requantized from int8 (superseded)</summary>
|
| 132 |
+
|
| 133 |
+
Before Comfy-Org published a pruned bf16 checkpoint, the only pruned source was
|
| 134 |
+
`pruned_int8_convrot`, so these files were built bf16 -> int8_convrot (Comfy-Org) -> NVFP4 —
|
| 135 |
+
two lossy passes whose error compounds. That path needed a real dequantize -> requantize,
|
| 136 |
+
because the StarNodes model converter passes non-floating-point tensors through untouched:
|
| 137 |
|
| 138 |
```python
|
| 139 |
+
if not tensor.dtype.is_floating_point:
|
| 140 |
+
return tensor # already-int8 weights are copied verbatim
|
|
|
|
|
|
|
|
|
|
|
|
|
| 141 |
```
|
| 142 |
|
| 143 |
+
so running it on an int8 checkpoint silently produced a byte-identical file. The old script
|
| 144 |
+
(`pruned_to_nvfp4.py`) is kept for reference. Both files were rebuilt single-pass on
|
| 145 |
+
2026-08-05; the output key set and dtype census are unchanged, only the error path is.
|
| 146 |
+
</details>
|
|
|
|
|
|
|
| 147 |
|
| 148 |
## Prompting: H3 wants a structured IR, not prose
|
| 149 |
|
|
|
|
| 227 |
VRAM (~20 GB vs ~12 GB) and can spend ~14% more time per step, Comfy-Org's
|
| 228 |
`pruned_int8_convrot` may be the better choice. This repo's value is smallest-footprint
|
| 229 |
and fastest, not highest-fidelity.
|
| 230 |
+
- **That observation predates the single-pass rebuild and has not been repeated since.** It
|
| 231 |
+
was made against the doubly-quantized files, so some of the smearing may have been
|
| 232 |
+
compounding error rather than 4-bit weights. Treat the int8-is-better lead as untested
|
| 233 |
+
against the current files.
|
| 234 |
- That comparison is **not controlled**: the two runs also differed in text encoder
|
| 235 |
(nvfp4_awq vs int8_convrot), it was n=1, and matched seeds do not produce matched
|
| 236 |
trajectories across different quantization. Treat it as a lead, not a result.
|
|
|
|
| 240 |
- 15s (362 frames) generates fine; see the VRAM note below.
|
| 241 |
- The **`fl2va` file is unbenchmarked** — it was produced by the same script over the same
|
| 242 |
layer set as `ref2va`, but has not been run end-to-end.
|
| 243 |
+
- Benchmarks are single-GPU, one card, one resolution, and were measured on the older
|
| 244 |
+
doubly-quantized files. Size and layout are unchanged by the rebuild, so s/it and VRAM
|
| 245 |
+
should carry over, but they have not been re-measured.
|
| 246 |
|
| 247 |
Failure cases are welcome in the discussions tab — concrete artifacts beat aggregate scores.
|
| 248 |
|