---
license:
- apache-2.0
- mit
base_model:
- Qwen/Qwen2.5-0.5B-Instruct
- Qwen/Qwen2.5-Coder-3B-Instruct
- HuggingFaceTB/SmolVLM-Instruct
language:
- en
datasets:
- tatsu-lab/alpaca
- databricks/databricks-dolly-15k
- teknium/OpenHermes-2.5
- stingning/ultrachat
metrics:
- accuracy
library_name: transformers
tags:
- text-generation
- custom-slm
- conversational
- agentic-router
- qlora
- chroma-rag
- local-ai
- pytorch
- vision
- multimodal
- code-generation
- vlm
- vqa
- ocr
---

# Zenthi-AI OS: Agentic Multi-Model Small Language Model (SLM)

Zenthi-AI is a production-grade, custom fine-tuned Small Language Model (SLM) conversational assistant. It is optimized for high-speed, local-first execution and acts as the intent-routing brain and synthesis engine of the **Zenthi-AI Multi-Model Operating System**.

This repository hosts the merged, full-precision model weights. 

---

## 🚀 Key Features

* **Base Foundation**: Built on the highly capable `Qwen/Qwen2.5-0.5B-Instruct`.
* **Parameter-Efficient Finetuning**: Optimized via QLoRA (4-bit quantization) on a merged, cleaned dataset of Alpaca, Dolly 15K, OpenHermes, UltraChat, and ShareGPT.
* **Agentic Orchestrator Routing**: Tuned specifically to act as a Router and Planner Agent, classifying query intents with high accuracy (CODE, VISION, RAG, SEARCH, KNOWLEDGE, COMPLEX).
* **Quantization-Ready**: Quantized to GGUF format for local deployment (quantized size under 500 MB).
* **Local RAG Integration**: Built to work in tandem with local ChromaDB embedding vector stores.
* **Web Search Coordination**: Designed to synthesize real-time context fetched from local SearXNG search clients.
* **Memory Management**: Keeps a windowed session history for conversational continuity.

---

## 📊 Evaluation & Routing Performance

The model's semantic routing accuracy was benchmarked across **500 unique evaluation test queries** (100 queries per intent category) running on a local GPU:

* **Overall Routing Accuracy**: **72.60%**
* **Average Latency**: **651.54 ms** per query

| Intent Category | Accuracy (%) | Target Expert Model |
| :--- | :--- | :--- |
| **CODE** | **100.00%** | `qwen2.5-coder:3b` |
| **VISION** | **100.00%** | `riven/smolvlm:latest` |
| **SEARCH** | **99.00%** | `qwen2.5:1.5b-instruct` |
| **RAG** | **43.00%** | `qwen2.5:1.5b-instruct` |
| **KNOWLEDGE** | **21.00%** | `qwen2.5:1.5b-instruct` |

---

## 💻 Local Usage & Integration

### 1. Ollama Deployment (GGUF)
To run Zenthi-AI locally in Ollama:

1. Create a `Modelfile` with the system prompt:
   ```dockerfile
   FROM zenthi-ai:latest
   PARAMETER temperature 0.7
   PARAMETER top_p 0.9
   SYSTEM """I am Zenthi-AI OS, a production-grade Agentic Multi-Model AI Operating System. I deliver accurate, secure, maintainable, and production-ready solutions by coordinating specialized AI capabilities."""
   ```
2. Build and run:
   ```bash
   ollama create Zenthi-AI -f Modelfile
   ollama run Zenthi-AI
   ```

### 2. Python Transformers API
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "KATHIR2006/zenthi-ai"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

# Start conversation
messages = [
    {"role": "system", "content": "You are Zenthi-AI OS, a production-grade Agentic Multi-Model AI Operating System."},
    {"role": "user", "content": "Explain photosynthesis simply."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to("cuda")

generated_ids = model.generate(**inputs, max_new_tokens=512)
response = tokenizer.batch_decode(generated_ids[:, inputs.input_ids.shape[1]:], skip_special_tokens=True)[0]
print(response)
```

---

## 🛠️ Fine-Tuned Expert Adapters

This repository also hosts the fine-tuned LoRA adapters for the specialized expert models of the Zenthi-AI OS:

### 1. Code Expert Adapters (`code-adapters/`)
* **Base Model**: `Qwen/Qwen2.5-Coder-3B-Instruct`
* **Dataset**: Custom programming and instruction dataset (1,200 training steps)
* **Final Loss**: **0.1843**
* **Usage**: Optimized for React, Node.js, Python, MERN stack development, reviews, and refactoring.

### 2. Vision Expert Adapters (`vision-adapters/`)
* **Base Model**: `HuggingFaceTB/SmolVLM-Instruct`
* **Dataset**: Synthetic VQA shape and color recognition dataset (100 training steps)
* **Final Loss**: **0.9077**
* **Usage**: Fine-tuned for OCR, visual question-answering, and image analysis.

---

## ⚖️ Licenses & Compliance

This project is dual-licensed:
* **LLM Model Weights & Adaptations**: Licensed under the **Apache License 2.0** (in compliance with the base Qwen2.5 license).
* **RAG Engine, Multi-Agent Framework, & Backend Codebase**: Licensed under the **MIT License**.