DeepSeek-R1 Full Series Local Deployment and High-Speed Weight Pulling (Ollama / vLLM / LMDeploy)

💡 DeepSeek-R1 architecture analysis and full-size selection

DeepSeek-R1 is a new generation of reinforcement learning-driven inference large model released by DeepSeek. Different from traditional post-training supervised fine-tuning (SFT), DeepSeek-R1 uses large-scale reinforcement learning (RL) to directly stimulate the chain-of-thought (CoT) capability of the model, demonstrating a level comparable to cutting-edge closed-source reasoning models in terms of mathematical proof, complex code generation, and multi-step logical deduction.

In order to meet the deployment needs of computing power environments of different sizes, DeepSeek officially not only open sourced the complete 671B MoE basic weight, but also open sourced 6 high-precision distilled models (Distilled Models) based on Qwen2.5 and LLaMA3 architecture:

⚡ Use HF-Mirror mirror to quickly pull model weights

Taking the most balanced DeepSeek-R1-Distill-Qwen-32B as an example, its original weight consists of 17 volume safetensors, with a total volume of about 65 GB. If you pull it directly through the official channel, it is easy to reset the long connection due to network jitter.

By configuring the HF_ENDPOINT environment variable, let huggingface-cli use the HF-Mirror accelerated distribution channel:

# 1. Set the mirror source and model cache root directory
export HF_ENDPOINT="https://hf-mirror.net"
export HF_HUB_ENABLE_HF_TRANSFER="0"

# 2. Start breakpoint resumable download to download the 32B distilled version
huggingface-cli download deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --local-dir /data/models/DeepSeek-R1-Distill-Qwen-32B \
  --local-dir-use-symlinks False \
  --resume-download

Tip: --local-dir-use-symlinks False will place the weights directly in the target directory to facilitate the smooth mounting of subsequent containers.

🚀 Three types of local inference deployment practices

📌 Option 1: Ollama runs locally at high speed out of the box

If you are an individual developer or testing verification, Ollama provides the most concise GGUF quantification operation method. After configuring the environment variables, Ollama can directly pull the corresponding quantized version:

# Start the 32B model (requires about 20GB video memory)
ollama run deepseek-r1:32b

# Or start the lightweight version 8B (requires about 5.5GB video memory)
ollama run deepseek-r1:8b

📌 Option 2: vLLM high-concurrency enterprise-level API service

vLLM is recommended for production environments. Using PagedAttention and Tensor Parallelism, single-card and multi-card throughput can be maximized:

# Dual card RTX 3090/4090 starts 32B distillation model (TP=2)
python3 -m vllm.entrypoints.openai.api_server \
  --model /data/models/DeepSeek-R1-Distill-Qwen-32B \
  --tensor-parallel-size 2 \
  --trust-remote-code \
  --gpu-memory-utilization 0.92 \
  --max-model-len 32768 \
  --port 8000 \
  --host 0.0.0.0

📌 Solution 3: LMDeploy ultra-fast inference and AWQ 4-bit compression

If you only have a single RTX 4090 with 24GB of video memory, you will face OOM (out of memory) when running the original 32B. At this time, using LMDeploy with AWQ 4-bit quantization is the optimal solution:

# Use the LMDeploy TurboMind engine to directly run the AWQ weighted version
lmdeploy serve api_server deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --backend turbomind \
  --cache-max-entry-count 0.85 \
  --model-format awq \
  --server-port 8000

⚠️ Production Pitfall Avoidance and Chain of Thinking (CoT) Label Analysis

1. Thinking process label truncation: DeepSeek-R1 will first output the ... logical chain and then the text. In front-end or business logic, it is recommended to divide the display area according to to give users an intuitive "thinking deeply" interactive experience.

2. Temperature coefficient (Temperature) tuning: Officials strongly recommend maintaining the inference Temperature between 0.5 ~ 0.7 (0.6 is recommended). Too high a sampling temperature can easily cause the inference chain logic to diverge or even fall into an infinite loop.

3. CUDA OOM troubleshooting: During long text inference, KV Cache memory usage increases linearly with the context length. --max-model-len should be clearly set when starting vLLM. Do not blindly enable the maximum 128k, which may cause the process to crash under sudden requests.