Mistral Large 2 & Pixtral 12B Multimodal: Tekken Tokenizer, Function Calling & vLLM Serving

🌪️ Tekken tokenizer and Pixtral visual architecture

Mistral AI's latest generation model introduces the Tekken tokenizer (the vocabulary size is expanded to 131k), which increases the Token compression rate by more than 30% when compressing multi-language and source code. Pixtral 12B uses the native ViT visual encoder to represent images as a series of 16x16 patch tokens, enabling high-precision visual question and answer on a single card with 24G video memory.

⚡ Image pulling and vLLM service startup

Pixtral 12B vLLM deployment

export HF_ENDPOINT="https://hf-mirror.net"

# Download Pixtral 12B Weights
huggingface-cli download mistralai/Pixtral-12B-2409 \
  --local-dir /data/models/Pixtral-12B \
  --local-dir-use-symlinks False

# Start the vLLM vision service
vllm serve /data/models/Pixtral-12B \
  --tokenizer_mode mistral \
  --max-model-len 16384 \
  --limit-mm-per-prompt image=4 \
  --port 8000