LLaMA 3.2 Vision: Cross-Attention Adapter Architecture & vLLM High-Throughput Serving

👁️ LLaMA 3.2 Vision Architecture: Cross-Attention Fusion

Meta did not fully retrain the language model base in LLaMA 3.2 Vision. Instead, it retained the pre-trained LLaMA 3.1 weight freeze, inserted Cross-Attention Layers between every several Transformer layers, and received the image representation output by the visual encoder. This structure not only protects the powerful reasoning ability of pure text, but also gives sophisticated understanding of charts and complex photos.

⚡ Carrying Token pulls the 11B visual version through the mirror

High-speed pulling of LLaMA 3.2 visual models

# Set up mirroring and read-only token
export HF_ENDPOINT="https://hf-mirror.net"
export HF_TOKEN="hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

# Mirror pull LLaMA 3.2 11B visual multi-modal
huggingface-cli download meta-llama/Llama-3.2-11B-Vision-Instruct \
  --local-dir /data/models/Llama-3.2-11B-Vision-Instruct \
  --local-dir-use-symlinks False \
  --resume-download