Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

32,838

Base only

Active filters: 8-bit

BigHuggyD/BigHuggyD-Grey-WizardLM-2-8x22B_exl2_8.0bpw_h8

Text Generation • Updated Jun 19, 2024 • 5

q18/alpindale-magnum-72b-v1-exl-8.0bpw-h8

Text Generation • Updated Jun 19, 2024 • 5

lucyknada/alpindale-magnum-72b-v1-8.0bpw

Text Generation • Updated Jun 19, 2024 • 6 • 1

RyanJT/quantized-granite-8bit-3b-code-instruct

Text Generation • 3B • Updated Jun 19, 2024 • 5

IEITYuan/Yuan2-M32-hf-int8

Text Generation • Updated Jun 20, 2024 • 9

sealai/sdxl-turbo-int8

Text-to-Image • 3B • Updated Jun 20, 2024 • 1

sealai/stable-diffusion-xl-base-1.0-int8

Text-to-Image • 3B • Updated Jun 20, 2024 • 1

ismailpubg5/v2_model

Text Generation • 7B • Updated Jun 20, 2024 • 8

bullerwins/L3-8B-Stheno-v3.2-exl2_8.0bpw

Text Generation • Updated Jun 20, 2024 • 4

booksouls/summarization-endpoint

0.4B • Updated Jun 26, 2024 • 3

Manu1507/Llama-2-7b-quantize

Text Generation • 7B • Updated Jun 21, 2024 • 4

SotiriosKastanas/lmsys-vicuna-7b-v1.5-8-bit-bnb

Text Generation • 7B • Updated Jun 21, 2024 • 6

SotiriosKastanas/lmsys-vicuna-13b-v1.5-8-bit-bnb

Text Generation • 13B • Updated Jun 21, 2024 • 10

ByteForge/Llama_3_8b_Instruct_Text2Sql_Finetuned

Text Generation • 8B • Updated Jun 21, 2024 • 9

NimaZahedinameghi/nimaAxolotl

Updated Jun 21, 2024 • 1

RyanJT/quantized-tinyllama-8bit-1.1b-chat2

Text Generation • 3B • Updated Jun 22, 2024 • 4

RyanJT/quantized-4bit-TinyLlama-1.1B-Chat-v1.0

Text Generation • 3B • Updated Jun 22, 2024 • 4

DFofanov78/llama-3-8b-Instruct-bnb-4bit

Text Generation • 5B • Updated Jun 22, 2024 • 3

Wsassi/whisper_minds_14_merged

Automatic Speech Recognition • 0.2B • Updated Jun 22, 2024 • 3

nm-testing/llama-3-instruct-w8a8-dyn-per-token-test

Text Generation • 8B • Updated Oct 9, 2024 • 10

bdambrosio/Hermes-2-Theta-Llama-3-70B-8.0bpw-exl2

Text Generation • Updated Jun 22, 2024 • 4

Sinensis/DarkForest-20B-v3.0-bpw8.0-h8-exl2

Text Generation • Updated Jun 26, 2025 • 5

SicariusSicariiStuff/LLAMA-3_8B_Unaligned_Alpha_RP_Soup_EXL2_8.0bpw

Text Generation • Updated Jul 25, 2025 • 4

Pyroserenus/L3-8B-Stheno-v3.3-32K-8.0bpw-h8-exl2

Text Generation • Updated Jun 24, 2024 • 4 • 3

Litzy619/MIS0623T1

4B • Updated Jun 23, 2024

SicariusSicariiStuff/LLAMA-3_8B_Unaligned_Alpha_EXL2_8.0bpw

Text Generation • Updated Jul 25, 2025 • 5

RyanJT/trained-quant-8bit

3B • Updated Jun 23, 2024 • 1

bigstorm/Hermes-2-Theta-Llama-3-8B-8.0bpw-8hb-exl2

Text Generation • Updated Jun 23, 2024 • 5

q18/sophosympatheia-New-Dawn-Llama-3-70B-32K-v1.0-exl2-8.0bpw-h8

Text Generation • Updated Jun 23, 2024 • 6 • 1

bigstorm/Codestral-22B-v0.1-8.0bpw-8hb-exl2

Text Generation • Updated Jun 23, 2024 • 4