---
license: apache-2.0
language:
- en
base_model:
- Qwen/Qwen3-Reranker-4B
pipeline_tag: text-ranking
library_name: transformers
tags:
- text-generation-inference
---

# **Qwen3-Reranker-4B-F32-GGUF**

> Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.


## Model Files

| Filename | Size | Format | Description |
|----------|------|--------|-------------|
| Qwen3-Reranker-4B.BF16.gguf | 8.05 GB | BF16 | Brain Float 16-bit quantization |
| Qwen3-Reranker-4B.F16.gguf | 8.05 GB | F16 | Half precision (16-bit) floating point |
| Qwen3-Reranker-4B.F32.gguf | 16.1 GB | F32 | Full precision (32-bit) floating point |
| Qwen3-Reranker-4B.Q2_K.gguf | 1.67 GB | Q2_K | 2-bit quantization with K-quant |
| Qwen3-Reranker-4B.Q3_K_L.gguf | 2.24 GB | Q3_K_L | 3-bit quantization (Large) with K-quant |
| Qwen3-Reranker-4B.Q3_K_M.gguf | 2.08 GB | Q3_K_M | 3-bit quantization (Medium) with K-quant |
| Qwen3-Reranker-4B.Q3_K_S.gguf | 1.89 GB | Q3_K_S | 3-bit quantization (Small) with K-quant |
| Qwen3-Reranker-4B.Q4_K_M.gguf | 2.5 GB | Q4_K_M | 4-bit quantization (Medium) with K-quant |
| Qwen3-Reranker-4B.Q4_K_S.gguf | 2.38 GB | Q4_K_S | 4-bit quantization (Small) with K-quant |
| Qwen3-Reranker-4B.Q5_K_M.gguf | 2.89 GB | Q5_K_M | 5-bit quantization (Medium) with K-quant |
| Qwen3-Reranker-4B.Q5_K_S.gguf | 2.82 GB | Q5_K_S | 5-bit quantization (Small) with K-quant |
| Qwen3-Reranker-4B.Q6_K.gguf | 3.31 GB | Q6_K | 6-bit quantization with K-quant |
| Qwen3-Reranker-4B.Q8_0.gguf | 4.28 GB | Q8_0 | 8-bit quantization |

### Recommended Usage for Reranking Tasks

- **Q4_K_M** or **Q5_K_M**: Optimal balance for most reranking applications
- **Q6_K** or **Q8_0**: Higher precision for critical ranking accuracy
- **Q3_K_M**: Good performance with reduced memory footprint
- **F16** or **BF16**: Maximum reranking precision, requires more VRAM
- **F32**: Highest precision for research and benchmarking

## Quants Usage 

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant
types (lower is better):

![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)