Instructions to use johan237/localgrammar-qwen25-3b-lora-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use johan237/localgrammar-qwen25-3b-lora-v1 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "johan237/localgrammar-qwen25-3b-lora-v1")

Transformers

How to use johan237/localgrammar-qwen25-3b-lora-v1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="johan237/localgrammar-qwen25-3b-lora-v1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("johan237/localgrammar-qwen25-3b-lora-v1", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use johan237/localgrammar-qwen25-3b-lora-v1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "johan237/localgrammar-qwen25-3b-lora-v1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "johan237/localgrammar-qwen25-3b-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/johan237/localgrammar-qwen25-3b-lora-v1

SGLang

How to use johan237/localgrammar-qwen25-3b-lora-v1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "johan237/localgrammar-qwen25-3b-lora-v1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "johan237/localgrammar-qwen25-3b-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "johan237/localgrammar-qwen25-3b-lora-v1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "johan237/localgrammar-qwen25-3b-lora-v1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Desktop
Docker Model Runner
How to use johan237/localgrammar-qwen25-3b-lora-v1 with Docker Model Runner:
```
docker model run hf.co/johan237/localgrammar-qwen25-3b-lora-v1
```

localgrammar-qwen25-3b-lora-v1

localgrammar-qwen25-3b-lora-v1 is a LoRA adapter for unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit specialized for local English rewriting.

It was fine-tuned for four controlled rewrite tasks:

App action	Training tag	Purpose
`grammar`	`TASK_GRAMMAR`	Fix grammar, spelling, punctuation, and agreement errors with minimal edits while preserving meaning and style.
`simplify`	`TASK_SIMPLIFY`	Make the text simpler and easier to understand while preserving meaning.
`clarity`	`TASK_CLARITY`	Make the text clearer and easier to understand while preserving meaning.
`coherence`	`TASK_COHERENCE`	Improve flow, structure, and coherence while preserving meaning.

There is no separate TASK_PHRASING tag. If you want a general phrasing rewrite, use:

TASK_CLARITY for sentence-level rewording
TASK_COHERENCE for flow and structure improvements

This repository contains the adapter weights and tokenizer assets. It is not a standalone base model.

Intended Use

This adapter is intended for:

local grammar correction
plain-English simplification
clarity-focused rewrites
coherence and flow edits
controlled rewriting in offline or local-first writing tools

It is not intended as a general-purpose open-ended chat assistant.

Prompt Format

The adapter expects the task to be stated explicitly in the user message.

During fine-tuning, each example used:

system prompt: You are a precise local writing engine. Follow the requested task exactly. Preserve the user's meaning.
user prompt format:

LANG_EN | TASK_...
Instruction: ...
Input: ...

Example for grammar correction:

LANG_EN | TASK_GRAMMAR
Instruction: Fix grammar, spelling, punctuation, and agreement errors with minimal edits. Preserve meaning and style.
Input: She go to school yesterday.

Quick Start

Load this adapter on top of the base model with PEFT.

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit"
adapter_id = "johan237/localgrammar-qwen25-3b-lora-v1"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    trust_remote_code=True,
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

messages = [
    {
        "role": "system",
        "content": (
            "You are a precise local writing engine. "
            "Follow the requested task exactly. Preserve the user's meaning."
        ),
    },
    {
        "role": "user",
        "content": (
            "LANG_EN | TASK_GRAMMAR\n"
            "Instruction: Fix grammar, spelling, punctuation, and agreement "
            "errors with minimal edits. Preserve meaning and style.\n"
            "Input: She go to school yesterday."
        ),
    },
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=False,
        temperature=0.0,
    )

generated_ids = output_ids[0][inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(generated_ids, skip_special_tokens=True))

Expected output:

She went to school yesterday.

How It Was Fine-Tuned

This adapter was fine-tuned with supervised fine-tuning on a chat-formatted English rewrite dataset assembled for local writing assistance.

Training data construction

The training set was built from:

grammarly/coedit
jhu-clsp/jfleg
facebook/asset
chaojiang06/wiki_auto

The dataset compiler:

mapped source tasks into the four task tags above
removed duplicate source/target pairs within each task
stripped instruction-like prefixes from some source examples
filtered obviously broken rows
filtered very short examples
filtered examples with large source/target length mismatches

Training set summary

total rows: 42,247
train rows: 42,035
validation rows: 2,212

Task distribution:

TASK_GRAMMAR: 20,000
TASK_SIMPLIFY: 15,000
TASK_COHERENCE: 8,000
TASK_CLARITY: 1,247

Dataset usage:

CoEdIT provided the main multi-task supervision
JFLEG augmented grammar correction
ASSET and WikiAuto augmented simplification

LoRA configuration

From adapter_config.json:

LoRA rank r=16
lora_alpha=32
lora_dropout=0
target modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj

Recorded training state

From the saved trainer state:

supervised fine-tuning
1 epoch
1855 total training steps
evaluation every 250 steps
save every 250 steps
recorded final validation loss: 0.4007
recorded train batch size: 2

Framework Versions

The exported training metadata reported:

PEFT 0.19.1
TRL 0.24.0
Transformers 5.5.0
PyTorch 2.5.1+cu121
Datasets 4.3.0
Tokenizers 0.22.2

Limitations

English only
optimized for short-to-medium rewrite requests rather than open-ended chat
output quality depends strongly on using the correct task tag
some training examples come from public rewrite datasets and may retain occasional dataset artifacts
TASK_CLARITY has much less training data than grammar or simplification

License

The base model Qwen/Qwen2.5-3B-Instruct is published on Hugging Face under the qwen-research license. This adapter should be used and redistributed in compliance with the upstream base-model license and the terms of the source datasets.

Sources

Base model family: Qwen/Qwen2.5-3B-Instruct
Quantized training base: unsloth/qwen2.5-3b-instruct-unsloth-bnb-4bit
Datasets:
- grammarly/coedit
- jhu-clsp/jfleg
- facebook/asset
- chaojiang06/wiki_auto

Citation

If you use this adapter, cite the base model, the source datasets, and your own project/repository in addition to standard library citations.

TRL citation:

@misc{vonwerra2022trl,
  title        = {TRL: Transformer Reinforcement Learning},
  author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
  year         = {2020},
  journal      = {GitHub repository},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/huggingface/trl}}
}

Downloads last month: 2

johan237
/

localgrammar-qwen25-3b-lora-v1