---
base_model: MiniMaxAI/MiniMax-H3
base_model_relation: quantized
library_name: diffusers
pipeline_tag: image-text-to-video
license: other
license_name: minimax-h3-community-license
license_link: LICENSE
tags:
  - auto-round
  - w4a16
  - int4
  - diffusion
  - video
  - minimax-h3
---

# MiniMax-H3 transformer, W4A16 (AutoRound RTN)

The joint video/audio diffusion transformer of
[MiniMaxAI/MiniMax-H3](https://huggingface.co/MiniMaxAI/MiniMax-H3), weight-only
quantized to 4 bits. **66.28 GB → 37.76 GB** (0.570 of the original).

This repository holds **only the transformer**. Every other component of the pipeline
(the Qwen3-VL conditioner, the video and audio VAEs, tokenizer, processor, both
schedulers) is unchanged and has to come from the original repository.

## What is quantized and what is not

| Group | Params | Precision |
|---|---|---|
| `transformer_blocks.*.attn.to_{q,k,v}`, `attn.to_out.0`, `ff.net.0.proj`, `ff.net.2` | 19.27 B (300 layers) | INT4, group 128, symmetric |
| `transformer_blocks.*.adaln_proj.linear` | 13.01 B (50 layers) | BF16 |
| `proj_in`, `proj_out`, `audio_proj_in`, `audio_proj_out`, `context_embedder`, `time_embedder`, `token_refiner.*`, `norm_out.linear` | ~0.7 B | BF16 |

`adaln_proj` is deliberately left alone, and it is the reason the file is 37.76 GB and
not ~12 GB. It is 13.0 B of the model's 33.1 B parameters, and quantizing it is a bad
trade: its input is the timestep embedding, whose row count is
`num_timesteps * MODALITY_NUM` rather than the token count, so it costs almost nothing
to compute no matter the precision. Meanwhile the diffusers implementation notes that a
rounding applied before its SiLU biases every block's modulation parameters identically
at every sampling step, so the error accumulates coherently along the denoising
trajectory instead of averaging out.

The keep-list for the remaining layers is the one the diffusers integration uses for its
own int8 recipe, and it matches what SGLang keeps in FP32 for this model: patch
projections, the timestep MLP, and the final video/audio heads.

## Honest limitations

* **This is RTN, not calibrated AutoRound.** `iters=0, disable_opt_rtn=True`, so no
  calibration data was used at all and quality is round-to-nearest, comparable to a
  GGUF `Q4_K_M` produced without an importance matrix. A calibrated run needs
  calibration inputs for a DiT (`hidden_states`, `temb`, `adaln_indices`, `rotary_emb`),
  which AutoRound's diffusion driver cannot produce for H3 today: it detects diffusion
  models by `model_index.json` and H3 ships `modular_model_index.json`, and its loader
  goes through `AutoPipelineForText2Image`, which has no H3 mapping.
* **No generated video has been compared against the BF16 baseline.** The BF16 pipeline
  needs far more accelerator memory than the machine this was produced on has, so there
  is no reference to diff against. What has been verified is that the weights are
  physically packed and that diffusers loads the checkpoint back.
* **Expect no speedup.** A video DiT pushes tens to hundreds of thousands of latent
  positions through every step, so its GEMMs are compute bound, and W4A16 there is a
  memory play rather than a throughput one. Speed on this class of model comes from
  quantizing activations too (FP8/W8A8 on Hopper or Blackwell), not from 4-bit weights.

## Loading

```python
import torch
from diffusers import AutoRoundConfig, MiniMaxH3Transformer3DModel, ModularPipeline

transformer = MiniMaxH3Transformer3DModel.from_pretrained(
    "Ar4ikov/MiniMax-H3-transformer-W4A16-RTN",
    quantization_config=AutoRoundConfig(backend="auto"),
    dtype=torch.bfloat16,
)

pipe = ModularPipeline.from_pretrained("MiniMaxAI/MiniMax-H3")
pipe.update_components(transformer=transformer)
pipe.load_components(dtype=torch.bfloat16)
```

MiniMax-H3 is not in a diffusers release yet, so this needs diffusers from its pull
request:

```
pip install "git+https://github.com/huggingface/diffusers.git@refs/pull/14355/head"
pip install "auto-round>=0.13.0"
```

For faster CUDA kernels also install `gptqmodel>=5.8.0` and pass
`AutoRoundConfig(backend="marlin")`.

## Recipe

```python
from auto_round import AutoRound

# layer_config maps every Linear outside the 6 targeted leaves of each block
# (and every adaln_proj.linear) to {"bits": 16, "act_bits": 16}
ar = AutoRound(
    model=transformer,              # MiniMaxH3Transformer3DModel, bf16
    scheme="W4A16",
    group_size=128,
    sym=True,
    layer_config=layer_config,
    to_quant_block_names="transformer_blocks",
    low_gpu_mem_usage=True,
    batch_size=1,
    iters=0,
    disable_opt_rtn=True,
)
ar.quantize_and_save(out_dir, format="auto_round", inplace=True)
```

Produced on 2x RTX 3090: 50 blocks in 16m53s, peak 13.9 GB host RAM and 4.05 GB VRAM,
since AutoRound onloads one block at a time.

Note that `quantize_and_save` raises `AttributeError: 'FrozenDict' object has no
attribute 'save_pretrained'` at the very end on this path: AutoRound's LLM export calls
`model.config.save_pretrained(...)`, which a diffusers config does not implement, and its
diffusion export path is not reached because H3 is not detected as a diffusion model. The
shards are already written when this happens; `config.json`, `quantization_config.json`
and the diffusers shard names were written afterwards, with the layer lists in
`extra_config` read back from the tensors actually present in the shards.

`auto_round` 0.14.2, `diffusers` 0.40.0.dev0 (PR 14355), `torch` 2.11.0+cu128.

## License and modifications

These weights are a modified version of MiniMax-H3 and stay under the **MiniMax H3
Community License Agreement**, a copy of which is included as `LICENSE`. `NOTICE` states
which files were modified and how, as the license requires. Anyone receiving these
weights receives them under that same agreement.

Two conditions in it are easy to miss and are on you as a user, not on this repository:
the license restricts use and distribution to its **Applicable Territory**, which excludes
the EU, the UK, South Korea and the USA; and products built on it must display
"Powered by MiniMax H3" and mark generated media with an AI-generation identifier.
Read `LICENSE` rather than taking this paragraph as a summary of it.