Tencent HunyuanVideo: Dual-Stream DiT Video Generation, 54GB Model Pulling & Low-VRAM Offload

🎬 Analysis of HunyuanVideo video generation architecture

Tencent HunyuanVideo has 13 billion parameters and uses a Diffusion Transformer design that mixes dual streams and single streams:

Dual-stream text-video attention: The early layer retains the independent expression of video frame Latent and multi-language text Token respectively, and the later layer is unified and integrated.
High-fidelity 3D causal VAE: 4x8x8 joint compression of the temporal and spatial dimensions, significantly reducing the DiT sequence length.

⚡ High-speed image pulling and Diffusers pipeline running

Download HunyuanVideo weight

# Image pull weight
export HF_ENDPOINT="https://hf-mirror.net"
huggingface-cli download tencent/HunyuanVideo \
  --local-dir /data/models/HunyuanVideo \
  --local-dir-use-symlinks False \
  --resume-download

video_gen.py: Single card 24G video memory offload video generation

import torch
from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel

transformer = HunyuanVideoTransformer3DModel.from_pretrained(
    "/data/models/HunyuanVideo", subfolder="transformer", torch_dtype=torch.bfloat16
)
pipe = HunyuanVideoPipeline.from_pretrained(
    "/data/models/HunyuanVideo", transformer=transformer, torch_dtype=torch.float16
)
# Enable video memory CPU offload, allowing 24GB single card to generate smoothly
pipe.enable_model_cpu_offload()
pipe.vae.enable_tiling()

output = pipe(
    prompt="A cute red panda wearing glasses reading a book in an ancient library, cinematic lighting, 4k.",
    num_frames=61,
    height=544,
    width=960,
    num_inference_steps=30
).frames[0]