--- license: apache-2.0 language: - en library_name: transformers pipeline_tag: text-generation base_model: Qwen/Qwen2.5-0.5B-Instruct tags: - agent - conversational - ollama - transformers - small-language-model - slm - tool-use - qwen - qwen2.5 - sakthai - house-of-sak - tool-calling - function-calling - merged - edge - lightweight - low-resource - raspberry-pi - on-device - benchmark - eval datasets: - Nanthasit/sakthai-combined-v7 - Nanthasit/sakthai-bench-v2 model-index: - name: sakthai-context-0.5b-tools results: - task: type: text-generation name: Tool-Calling dataset: name: SakThai Bench v2 (500 rows, scorer multiset-selection-v2) type: Nanthasit/sakthai-bench-v2 metrics: - type: selection value: 91.2 name: Selection Accuracy verified: true evidence: .eval_results/sakthai-bench-v2.yaml - type: arguments value: 45.7 name: Arguments Accuracy verified: true evidence: .eval_results/sakthai-bench-v2.yaml - type: strict value: 45.7 name: Strict Accuracy verified: true evidence: .eval_results/sakthai-bench-v2.yaml - type: held-out value: 87.8 name: Held-Out Tool Accuracy verified: true evidence: .eval_results/sakthai-bench-v2.yaml - type: degenerate value: 0 name: Degenerate Outputs verified: true evidence: .eval_results/sakthai-bench-v2.yaml inference: parameters: temperature: 0.01 max_new_tokens: 256 top_p: 0.9 widget: - text: What is the weather in Tokyo? example_title: Tool-calling - text: Who wrote Romeo and Juliet? example_title: Direct answer - text: Search the web for latest AI news example_title: Search tool ---

Downloads Size Pipeline License Collection Family

SakThai Context 0.5B Tools

Ultra-light tool-calling agent · Qwen2.5-0.5B fine-tune · runs in ~1 GB RAM

**SakThai Context 0.5B Tools** is a prompt-masked SFT of [Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) optimized for browser/tool calling. It achieves **91.2% selection accuracy** on SakThai Bench v2, with **0% degenerate outputs** in multi-trial evaluation. ## Model Description **SakThai Context 0.5B Tools** is a prompt-masked supervised fine-tune of [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) focused on reliable tool/function calling in conversational agents. The model is trained to select the correct tool, generate valid JSON-style arguments, and avoid degenerate outputs. It is optimized for edge deployment and can run on consumer hardware with ~1 GB RAM. Key points: - Base: `Qwen/Qwen2.5-0.5B-Instruct` - Training: prompt-masked SFT on tool-calling traces from `Nanthasit/sakthai-combined-v7` - Primary use: lightweight agents, on-device assistants, Raspberry Pi / edge deployments - License: Apache-2.0 ## Quick Start — Transformers ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "Nanthasit/sakthai-context-0.5b-tools" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto") tools = [ { "type": "function", "function": { "name": "get_weather", "description": "Get current weather", "parameters": { "type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"], }, } } ] messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the weather in Tokyo?"}, ] text = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True) inputs = tokenizer([text], return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.01, top_p=0.9) response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True) print(response) ``` ## Quick Start — llama.cpp / Ollama ```bash # Convert with llama.cpp and run locally llama-quantize ./sakthai-context-0.5b-tools-f16.gguf ./model-q4_k_m.gguf Q4_K_M ollama create sakthai-context-0.5b-tools -f Modelfile ollama run sakthai-context-0.5b-tools ``` ### Usage notes - For tool calling, always use `apply_chat_template(..., tools=tools, tokenize=False, add_generation_prompt=True)` so the model receives the proper `` block. - If you want stricter outputs, reduce `temperature` further, e.g. `0.0`. - For CPU-only inference, set `device_map="cpu"`; GPU/MPS/CPU auto-detection works with `device_map="auto"`. ## Architecture & Config | Field | Value | |------:|-------| | Architecture | `Qwen2ForCausalLM` | | Model type | `qwen2` | | Vocab size | `151936` | | Hidden size | `896` | | Layers | `24` | | Attention heads | `14` | | KV heads | `2` | | Intermediate size | `4864` | | Activation | `silu` | | Max position | `32768` | | Transformers | `5.14.1` | ## Benchmarks | Metric | Value | Verified | |------:|------:|:--------| | Selection Accuracy | 91.2% | true | | Arguments Accuracy | 45.7% | true | | Strict Accuracy | 45.7% | true | | Held-Out Tool Accuracy | 87.8% | true | | Degenerate Outputs | 0% | true | Evidence: `.eval_results/sakthai-bench-v2.yaml` in repo. ## Limitations - 0.5B parameter scale limits reasoning depth; arguments accuracy is lower than selection accuracy. - Tool schema adherence degrades on nested arguments and long context traces. - Current weights are merged; if you need the unmerged adapter, use `Nanthasit/sakthai-context-0.5b-tools-sft` or `Nanthasit/sakthai-context-0.5b-tools-sft-v2`. ## Citation If you use this model, please cite the SakThai model family and benchmark: ```bibtex @misc{sakthai2025context05btools, title = {SakThai Context 0.5B Tools}, author = {Nanthasit}, year = {2026}, url = {https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools} } ``` ## SakThai Family | Repo | Downloads | Size | Pipeline | |-----:|----------:|-----:|---------| | [Nanthasit/sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1855 | ~4.07 GB | text-generation | | [Nanthasit/sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 1692 | ~1.39 GB | text-generation | | [Nanthasit/sakthai-context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 1024 | ~15.23 GB | text-generation | | [Nanthasit/sakthai-embedding-multilingual](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) | 627 | ~471 MB | sentence-similarity | | [Nanthasit/sakthai-context-7b-128k](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) | 610 | — | text-generation | | [Nanthasit/sakthai-context-7b-tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) | 489 | ~20 MB | text-generation | | [Nanthasit/sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) | 477 | ~8.7 MB | text-generation | | [Nanthasit/sakthai-context-1.5b-merged-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) | 337 | ~3.09 GB | text-generation | | [Nanthasit/sakthai-vision-7b](https://huggingface.co/Nanthasit/sakthai-vision-7b) | 315 | ~4.71 GB | image-text-to-text | | [Nanthasit/sakthai-plus-1.5b-lora](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | 306 | ~74 MB | text-generation | | [Nanthasit/sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | 251 | ~1.0 GB | text-generation | | [Nanthasit/sakthai-tts-model](https://huggingface.co/Nanthasit/sakthai-tts-model) | 248 | ~143 MB | text-to-speech | | [Nanthasit/sakthai-plus-1.5b](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) | 244 | ~3.09 GB | text-generation | | [Nanthasit/sakthai-context-1.5b-tools-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | 173 | ~74 MB | text-generation | | [Nanthasit/sakthai-coder-1.5b](https://huggingface.co/Nanthasit/sakthai-coder-1.5b) | 151 | ~1.12 GB | text-generation | | [Nanthasit/sakthai-coder-browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) | 54 | ~3.09 GB | text-generation | | [Nanthasit/sakthai-coder-browser-gguf](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) | 35 | ~7.11 GB | text-generation | | [Nanthasit/sakthai-embedding](https://huggingface.co/Nanthasit/sakthai-embedding) | 23 | ~110 MB | sentence-similarity | | [Nanthasit/sakthai-coder-browser-lora](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) | 21 | ~74 MB | text-generation | Download counts and sizes were verified from the Hub API at upload time.