FLUX.1 Diffusion Model: ComfyUI Workflows, Schnell/Dev Weights & NF4 Low-VRAM Setup

🎨 FLUX.1 architecture and multi-component teardown

Black Forest Labs, built by the original founding team of Stable Diffusion, launched the FLUX.1 series of open source diffusion models. It has reached the industry benchmark level in terms of picture texture, finger body accuracy, typography and complex prompt understanding.

FLUX.1 consists of two main open source branches:

FLUX.1 [dev]: 12 billion parameter non-commercial high-precision base model, supports complex sampling steps (usually 20~30 steps), and has an extremely high upper limit of image quality.
FLUX.1 [schnell]: A rapid version for real-time generation of distillation, which can produce graphs in just 4 steps and follows the Apache 2.0 open source commercial license.

In project implementation (especially ComfyUI and WebUI), FLUX.1 is not a single file, but composed of multiple core components:

1. DiT Backbone (Diffusion Transformer)

The core network responsible for noise reduction and denoising, the original single file is about 23.8 GB (FP16) or 11.9 GB (FP8).

2. Dual text encoder (CLIP-L + T5-v1.1-XXL)

CLIP-L is responsible for basic semantics (about 240MB), and T5-XXL is responsible for extremely fine-grained text descriptions and long prompt words (FP16 is about 9.8GB, FP8 is about 4.9GB).

3. Variational Autoencoder (AE / VAE)

Responsible for decoding mapping between Latent latent space and high-resolution pixel images (single file is about 335MB).

4. LoRA style tweak adapter

The volume of low-rank adaptive weights mounted on the DiT linear layer is usually between 20MB ~ 300MB.

⚡ Mirror station modular pull and ComfyUI directory mapping

In order to avoid pulling too many useless files in ComfyUI, it is recommended to use huggingface-cli with the --include syntax to download on demand and directly drop it into the corresponding model directory of ComfyUI:

Automatically pull and archive to ComfyUI directory

#!/usr/bin/env bash
export HF_ENDPOINT="https://hf-mirror.net"
COMFY_ROOT="/data/ComfyUI"

echo "=== 1. Pull FLUX.1 VAE weight ==="
huggingface-cli download black-forest-labs/FLUX.1-dev ae.safetensors \
  --local-dir "$COMFY_ROOT/models/vae" \
  --local-dir-use-symlinks False

echo "=== 2. Pull CLIP-L text encoder ==="
huggingface-cli download comfyanonymous/flux_text_encoders clip_l.safetensors \
  --local-dir "$COMFY_ROOT/models/clip" \
  --local-dir-use-symlinks False

echo "=== 3. Pull T5XXL text encoder (FP8 low memory version recommended) ==="
huggingface-cli download comfyanonymous/flux_text_encoders t5xxl_fp8_e4m3fn.safetensors \
  --local-dir "$COMFY_ROOT/models/clip" \
  --local-dir-use-symlinks False

echo "=== 4. Pull FLUX.1-dev backbone network (FP8 optimized version) ==="
huggingface-cli download Kijai/flux-fp8 flux1-dev-fp8.safetensors \
  --local-dir "$COMFY_ROOT/models/unet" \
  --local-dir-use-symlinks False

echo "All components have been downloaded! The corresponding node can be loaded directly in ComfyUI."

🎛️ Video memory grading and low video memory startup tuning

FLUX.1 original FP16 operation requires more than 32GB of video memory. Personal workstations need to properly plan the accuracy and memory offloading strategy:

Hardware environment	Model format scheme	ComfyUI startup parameters	Image rendering speed (1024x1024)
12GB video memory (3060/4070)	NF4 / GGUF Q4_K_S + T5 FP8	--lowvram	About 30 ~ 45 seconds
16GB video memory (4080/4070TiS)	FP8 DiT + T5 FP8	--gpu-only or default	About 15 ~ 20 seconds
24GB video memory (3090/4090)	FP8 / Original FP16 + T5 FP8	--highvram	About 8 ~ 12 seconds

Tip: Use the DualCLIPLoader node in the ComfyUI workflow, select clip_l.safetensors and t5xxl_fp8_e4m3fn.safetensors respectively, and set the type to flux for perfect adaptation.