---
language:
- ku
- kmr
base_model:
- Qwen/Qwen2.5-7B-Instruct
- muzaffercky/Sorjin1-7B
tags:
- kurdish
- kurmanji
- instruct
- merge
- mergekit
- qwen2.5
license: apache-2.0
datasets:
  - muzaffercky/kurdish-kurmanji-theses
---

# Sorjîn1-7B-Instruct

Sorjîn1-7B-Instruct is a Kurdish Kurmanji instruction-following model produced by merging [Sorjîn1-7B](https://huggingface.co/muzaffercky/Sorjin1-7B) with [Qwen/Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) using [mergekit](https://github.com/arcee-ai/mergekit).

It combines Kurdish language knowledge from continual pretraining with the instruction-following capability of Qwen2.5-7B-Instruct.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "muzaffercky/Sorjin1-7B-Instruct",
    torch_dtype="bfloat16",
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("muzaffercky/Sorjin1-7B-Instruct")

messages = [{"role": "user", "content": "Tu çawa yî?"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0], skip_special_tokens=True))
```

## Merge Details

- **Method:** TIES merge
- **Models:** Sorjîn1-7B (Kurdish base) + Qwen/Qwen2.5-7B-Instruct
- **Interpolation factor:** 0.5 (equal weight)
- **Density:** 0.7
- **Tool:** [mergekit](https://github.com/arcee-ai/mergekit)

## Limitations

- Kurdish language coverage is limited to formal academic style — trained on university theses
- May mix languages (Kurdish/English/Turkish) on topics not covered in training data
- Not aligned with RLHF — use with appropriate safety considerations

## Model Family

| Model | Description |
|-------|-------------|
| [Sorjîn1-LoRA](https://huggingface.co/muzaffercky/Sorjin1-LoRA) | Raw LoRA adapter weights |
| [Sorjîn1-7B](https://huggingface.co/muzaffercky/Sorjin1-7B) | Continually pretrained base model |
| Sorjîn1-7B-Instruct (this model) | Merged with Qwen2.5-7B-Instruct |