Qwen2.5-Coder: Local Setup with Continue & Cline Plugins for Private Code Intelligence

💻 Why choose Qwen2.5-Coder as the main local code model?

Qwen2.5-Coder is a large code model specially designed for program design fine-tuning by Tongyi Lab, covering full sizes from 0.5B to 32B. The 32B version has scored higher than many 100-billion-level universal models on core lists such as EvalPlus, Aider, and HumanEval. Its core advantages are:

128K ultra-long context window: Ability to feed the entire project's code repository directory, multi-file dependencies, and API specifications into the model at once.
Fill-in-the-Middle (FIM) support: Perfectly matches IDE inline code completion (Tab Autocomplete) scenarios.

⚡ Image pull 32B weight and vLLM local deployment

Download and launch commands

# Set up mirroring
export HF_ENDPOINT="https://hf-mirror.net"

# Pull the fine-tuned version of the 32B instruction
huggingface-cli download Qwen/Qwen2.5-Coder-32B-Instruct \
  --local-dir /data/models/Qwen2.5-Coder-32B-Instruct \
  --local-dir-use-symlinks False

# Start vLLM OpenAI compatible interface (single card or dual card 24G running AWQ quantized version)
vllm serve /data/models/Qwen2.5-Coder-32B-Instruct \
  --tensor-parallel-size 2 \
  --max-model-len 32768 \
  --port 8000

🔌 Configuring Continue and Cline plugins

Append the local interface in ~/.continue/config.json of the VS Code plug-in Continue:

Continue configuration example

{
  "models": [
    {
      "title": "Qwen2.5-Coder-32B (Local)",
      "provider": "openai",
      "model": "Qwen2.5-Coder-32B-Instruct",
      "apiBase": "http://localhost:8000/v1",
      "apiKey": "EMPTY"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5-Coder-1.5B (FIM)",
    "provider": "ollama",
    "model": "qwen2.5-coder:1.5b-base"
  }
}