---
license: apache-2.0
tags:
- dspark
- speculators
- glm
name: GLM-5.2-speculator.dspark
---

# RedHatAI/GLM-5.2-speculator.dspark

This is a preliminary (and subject to change) [DSpark](https://github.com/vllm-project/speculators) speculator model for [RedHatAI/GLM-5.2-NVFP4-FP8](https://huggingface.co/RedHatAI/GLM-5.2-NVFP4-FP8 ).

It was trained using the [Speculators](https://github.com/vllm-project/speculators) library on the [mgoin/open-perfectblend-glm5.2-regen](https://huggingface.co/datasets/mgoin/open-perfectblend-glm5.2-regen) dataset.

## Note:

It was validated on Nvidia B200, other hardware validation pending.   

## Training Details

### Launch training

```bash
scripts/train.py \
  --verifier-name-or-path RedHatAI/GLM-5.2-NVFP4-FP8 \
  --data-path ./output/dspark_glm52 \
  --vllm-endpoint http://localhost:8000/v1 \
  --save-path ./output/dspark_glm52/checkpoints \
  --epochs 1 \
  --lr 3e-4 \
  --total-seq-len 8192 \
  --speculator-type dspark \
  --block-size 8 \
  --max-anchors 1024 \
  --num-layers 3 \
  --target-layer-ids 2 20 39 58 75 \
  --markov-rank 256 \
  --markov-head-type vanilla \
  --enable-confidence-head \
  --confidence-head-with-markov \
  --loss-fn '{"ce": 0.1, "tv": 0.9}' \
  --confidence-head-alpha 1.0 \
  --fsdp-shard \
  --logger tensorboard \
  --log-dir ./output/dspark_glm52/logs \
  --on-missing generate \
  --on-generate delete \
  --checkpoint-freq 0.01 \
  --request-timeout 60 \
  --max-retries 2
```

## Deployment

Deploy with vLLM using the speculator as a draft model.

```bash
vllm serve RedHatAI/GLM-5.2-NVFP4-FP8  \
  --max-model-len 20480 \
  --spec-model RedHatAI/GLM-5.2-speculator.dspark \
  --spec-method dspark \
  --spec-tokens 7 \
  -tp 8
```

## Preliminary Evaluations

Per-Position Acceptance Rate

| Dataset | Pos 0 | Pos 1 | Pos 2 | Pos 3 | Pos 4 | Pos 5 | Pos 6 | Avg. Length |
|:-------|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|:----------:|
| HumanEval | 77.8% | 59.1% | 44.6% | 33.9% | 26.1% | 20.3% | 15.8% | 3.78 |
| math_reasoning | 87.5% | 76.6% | 67.6% | 59.2% | 51.8% | 45.1% | 39.5% | 5.27 |
| qa | 70.7% | 48.5% | 33.7% | 23.7% | 17.1% | 11.9% | 8.4% | 3.14 |
| question | 73.7% | 52.6% | 38.1% | 28.5% | 21.6% | 16.7% | 13.2% | 3.44 |
| rag | 78.5% | 61.3% | 48.7% | 38.9% | 30.7% | 24.2% | 19.2% | 4.01 |
| summarization | 77.9% | 58.0% | 43.0% | 31.7% | 22.8% | 16.2% | 11.4% | 3.61 |
| tool_call | 71.9% | 51.1% | 37.1% | 27.6% | 21.4% | 16.8% | 13.2% | 3.39 |
| translation | 76.8% | 57.4% | 43.5% | 33.6% | 26.1% | 20.0% | 15.1% | 3.72 |
| writing | 74.0% | 52.2% | 38.0% | 28.1% | 21.3% | 16.7% | 13.1% | 3.44 |