---
pipeline_tag: text-generation
library_name: transformers
tags:
- speculative-decoding
- draft-model
- eagle3
- llama
- inference-acceleration
base_model: meta-llama/Llama-3.1-8B-Instruct
license: cc-by-4.0
model-index:
- name: nebius/EAGLE3-Llama-3.1-8B-Instruct
  results:
  - task:
      type: text-generation
    dataset:
      name: MT-Bench
      type: MT-Bench
    metrics:
    - name: Acceptance Length
      type: Acceptance Length
      value: 3.48
  - task:
      type: text-generation
    dataset:
      name: GSM8K
      type: GSM8K
    metrics:
    - name: Acceptance Length
      type: Acceptance Length
      value: 4.02
  - task:
      type: text-generation
    dataset:
      name: HumanEval
      type: HumanEval
    metrics:
    - name: Acceptance Length
      type: Acceptance Length
      value: 4.52
datasets:
- nebius/Llama-3.1-8B-Instruct-Infinity-Instruct-0625
---

## Model Description

This is an EAGLE-3 draft model for **Llama-3.1-8B-Instruct**, trained from scratch using **LK losses** — training objectives that directly target acceptance rate rather than using KL divergence as a proxy.

## Training Details

- **Base model**: meta-llama/Llama-3.1-8B-Instruct
- **Draft architecture**: EAGLE-3
- **Training data**: [Infinity-Instruct-0625 with Llama-3.1-8B generated responses](https://huggingface.co/datasets/nebius/Llama-3.1-8B-Instruct-Infinity-Instruct-0625)
- **Training objective**: Hybrid LK loss with adaptive λ scheduling (η=3)
- **Training**: 10 epochs from random initialization
- **Draft length**: K = 6 speculative tokens

## Performance

Average acceptance length (τ) measured across MT-bench, HumanEval, and GSM8K with K = 7:

| Configuration | Temperature = 0 | Temperature = 1 |
|---------------|-----------------|-----------------|
| EAGLE-3 + KL | 4.36 | 3.86 |
| **EAGLE-3 + LK (ours)** | **4.43** | **4.01** |


### Comparison with Public Checkpoints

| Model | MT-bench (τ) | HumanEval (τ) | GSM8K (τ) |
|-------|--------------|---------------|-----------|
| yuhuili/EAGLE3-LLaMA3.1-Instruct-8B | 2.97 | 3.75 | 3.21 |
| zhuyksir/EAGLE3-Llama-3.1-8B-Instruct | 3.06 | 3.96 | 3.21 |
| **Ours** | **3.48** | **4.52** | **4.02** |

*Measured at temperature = 1 with K = 7*

> **Note:** Earlier vLLM versions sampled draft tokens greedily regardless of temperature, which underestimated acceptance rates at temperature > 0. Stochastic draft sampling was introduced in **v0.18.0**, and from **v0.21.0** it can be enabled via `speculative_config` using `rejection_sample_method` and `draft_sample_method`. The acceptance metrics reported above were measured under standard rejection sampling and are reproducible with the configuration below.

## Usage with vLLM
```python
from vllm import LLM, SamplingParams

llm = LLM(
    model="meta-llama/Llama-3.1-8B-Instruct",
    speculative_config={
        "method": "eagle3",
        "model": "nebius/EAGLE3-Llama-3.1-8B-Instruct",
        "num_speculative_tokens": 6,
        "rejection_sample_method": "standard",
        "draft_sample_method": "gumbel",
    },
)

sampling_params = SamplingParams(temperature=0.7)
outputs = llm.generate(["Explain speculative decoding in simple terms."], sampling_params)
```

## License

[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

This model was trained using outputs from [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct). Use of this model is additionally subject to the [Llama 3.1 Community License Agreement](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE).

Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

## Citation

```
@misc{samarin2026lklosses,
  title     = {LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding},
  author    = {Alexander Samarin and Sergei Krutikov and Anton Shevtsov and Sergei Skvortsov and Filipp Fisin and Alexander Golubev},
  year      = {2026},
  eprint    = {2602.23881},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url       = {https://arxiv.org/abs/2602.23881}
}
```