---
license: apache-2.0
license_link: https://huggingface.co/tencent/Hy3/blob/main/LICENSE
base_model: tencent/Hy3
base_model_relation: quantized
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- oq
- oqe
- imatrix
- quantized
- moe
- hunyuan
- 2.31bpw
---

# Hy3 (MLX, oQ2e @ 2.31 bpw)

Calibrated 2-bit MLX quantization of tencent/Hy3 (Hunyuan 3.0, 295B-A21B MoE),
produced with omlx oQe at level 2. Effective 2.31 bits/weight over quantizable
weights, 85.3 GB on disk. Targets Apple Silicon.

This is the most **shell-reduced** variant of mlx-community/Hy3-oQ2e: the routed
experts are identical, but the non-expert layers (attention, embeddings, lm_head)
are quantized to 3-bit — the smallest of the oQ2e-derived ladder.

## Quantization layout

| Component            | oQ2e (parent)         | This model (2.31 bpw) |
|----------------------|-----------------------|-----------------------|
| Routed experts (98%) | 2-bit gs128 + imatrix | 2-bit gs128 + imatrix |
| Attention            | 8-bit gs64            | **3-bit gs128**       |
| Embeddings / lm_head | 8-bit gs64            | **3-bit gs128**       |

Routed experts are bit-for-bit the same as oQ2e (imatrix reused from the same
calibration cache). Only the shell changed.

## How it was quantized

Built from the BF16 source (~550 GB / 591 GiB) with omlx oQ level 2 + imatrix
weighting. To stay within 128 GB RAM: the sensitivity pass reused the existing
oQ2 quantization instead of building a full-precision proxy, and the importance
matrix was reused from the oQ2e calibration cache. Streamed tensor-by-tensor.

## Requirements

Runs natively in oMLX. For mlx-lm, hy_v3 support is pending upstream; until then:

    uv pip install "mlx-lm @ git+https://github.com/kernelpool/mlx-lm.git@add-hy3-preview"

## Benchmarks (all Hy3 MLX variants)

oMLX intelligence suite, 300 seeded samples per benchmark, identical questions
across models. I ran seeded samples — this is **not** a complete benchmark run,
so read the differences as noise and test the versions against your own workload
before picking one.

| Benchmark (300) | oQ2 · 2.68 | oQ2e · 2.43 | oQ2e-2.37bpw | oQ2e-2.33bpw | **oQ2e-2.31bpw (this model)** |
|---|---|---|---|---|---|
| mathqa     | 0.63 | 0.65 | 0.64 | 0.62 | 0.60 |
| mmlu_pro   | 0.65 | 0.61 | 0.60 | 0.59 | 0.55 |
| winogrande | 0.74 | 0.68 | 0.68 | 0.65 | 0.65 |

Variants: [oQ2](https://huggingface.co/mlx-community/Hy3-oQ2) ·
[oQ2e](https://huggingface.co/mlx-community/Hy3-oQ2e) ·
[oQ2e-2.37bpw](https://huggingface.co/mlx-community/Hy3-oQ2e-2.37bpw) ·
[oQ2e-2.33bpw](https://huggingface.co/mlx-community/Hy3-oQ2e-2.33bpw) ·
[oQ2e-2.31bpw](https://huggingface.co/mlx-community/Hy3-oQ2e-2.31bpw)

## Usage

    python -m mlx_lm generate --model mlx-community/Hy3-oQ2e-2.31bpw \
      --prompt "Explain Bayes' theorem in two sentences." --max-tokens 300

```python
from mlx_lm import load, generate
model, tokenizer = load("mlx-community/Hy3-oQ2e-2.31bpw")
```

## License

Apache-2.0, inherited from tencent/Hy3.