Ephraimmm/Pidgn_Rufus_model

Pidgn Rufus Model

A Llama 3.2 3B Instruct fine-tune exported as a single-file GGUF (F16) checkpoint, intended for conversational use in Nigerian Pidgin English (pcm).

Overview

This repository packages a fine-tuned version of Meta's Llama 3.2 3B Instruct model, converted to the GGUF format for use with llama.cpp-based runtimes (e.g. Ollama, LM Studio, llama.cpp itself). Based on the repository name and the included Ollama Modelfile, the model is intended to generate and respond to text in Nigerian Pidgin English, an English-based creole widely spoken in Nigeria.

The training data, dataset composition, and evaluation results are not published in this repository, so no claims are made here beyond what can be verified from the files present (config.json, Modelfile, README.md).

Training Details

Detail	Value
Base model	meta-llama/Llama-3.2-3B-Instruct (verified via Modelfile and architecture in config.json: LlamaForCausalLM, hidden size 3072, 28 layers, 24 attention heads, 8 KV heads, vocab size 128,256)
Fine-tuning framework	Unsloth (evidenced by unsloth_version and unsloth_fixed fields in config.json)
Final training loss	0.7410 (as reported in the prior model card)
Export format	GGUF, F16 precision, merged single-file checkpoint (llama-3.2-3b-instruct.F16.gguf)

Hyperparameters such as dataset size/source, number of epochs or steps, learning rate, and evaluation metrics are not included in this repository (no trainer_state.json, training_args.bin, or dataset card are present), so they are intentionally omitted rather than estimated.

Intended Use

• Conversational text generation in Nigerian Pidgin English (and English, inherited from the base model).
• Local/offline inference via GGUF-compatible runtimes such as Ollama or llama.cpp.
• Experimentation and prototyping for Pidgin-language chatbots, assistants, or translation-adjacent tools.

This model is a fine-tune of an instruct model and has not been evaluated for production, safety-critical, or high-stakes deployments.

How to Use

Option 1: Ollama

A ready-made Modelfile is included in this repository (built on the Llama 3 chat template with <|start_header_id|>/<|eot_id|> tokens). To run locally with Ollama:

# Download the gguf and Modelfile from this repo, then:
ollama create pidgn-rufus -f Modelfile
ollama run pidgn-rufus

Option 2: llama.cpp

./llama-cli -m llama-3.2-3b-instruct.F16.gguf -p "How you dey today?"

Option 3: transformers (GGUF loader)

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Ephraimmm/Pidgn_Rufus_model"
filename = "llama-3.2-3b-instruct.F16.gguf"

tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
model = AutoModelForCausalLM.from_pretrained(model_id, gguf_file=filename)

inputs = tokenizer("How you dey today?", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Option 4: Use our chat interface

"""
Interactive Nigerian Pidgin chat with Ephraimmm/Pidgn_Rufus_model (GGUF via transformers).

Fixes vs. the naive version:
  1. Applies the Llama-3.2 chat template (with fallback if GGUF carries no template)
  2. Stops on <|eot_id|>, not just <|end_of_text|>
  3. Slices the prompt off the generated ids so you only decode the reply
  4. Keeps conversation history for real back-and-forth
"""

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "Ephraimmm/Pidgn_Rufus_model"
GGUF_FILE = "llama-3.2-3b-instruct.F16.gguf"

SYSTEM_PROMPT = """You be Rufus, a Nigerian person wey dey yarn Naija Pidgin English.

Hard rules:
- Reply ONLY in Nigerian Pidgin English. No Standard English, no translation, no glossary.
- Talk DIRECTLY to the person, like say una dey gist face to face.
- NEVER describe, narrate, analyse or explain the conversation. Do not write things like
  "The speaker is asking..." or "This conversation is informal...". If you catch yourself
  explaining, stop and just answer instead.
- No stage directions, no asterisks, no roleplay actions.
- Answer for 1 to 3 short sentences unless the person ask for plenty detail.
- Never write the person own reply for dem. Answer your own turn, then stop."""

# --------------------------------------------------------------------------- load

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, gguf_file=GGUF_FILE)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    gguf_file=GGUF_FILE,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto" if torch.cuda.is_available() else None,
)
model.eval()

if tokenizer.pad_token_id is None:
    tokenizer.pad_token = tokenizer.eos_token

# Stop on BOTH end-of-turn and end-of-text. Missing <|eot_id|> is the #1 cause of
# the model rambling into a fake second turn.
terminators = {tokenizer.eos_token_id}
for tok_str in ("<|eot_id|>", "<|end_of_text|>"):
    tid = tokenizer.convert_tokens_to_ids(tok_str)
    if isinstance(tid, int) and tid >= 0 and tid != tokenizer.unk_token_id:
        terminators.add(tid)
terminators = list(terminators)

# --------------------------------------------------------------------- prompt build

LLAMA3_HEADER = "<|start_header_id|>{role}<|end_header_id|>\n\n{content}<|eot_id|>"


def _manual_llama3_prompt(messages):
    """Fallback if the GGUF metadata carries no chat_template."""
    out = "<|begin_of_text|>"
    for m in messages:
        out += LLAMA3_HEADER.format(role=m["role"], content=m["content"].strip())
    out += "<|start_header_id|>assistant<|end_header_id|>\n\n"
    return out


def build_inputs(messages):
    if getattr(tokenizer, "chat_template", None):
        text = tokenizer.apply_chat_template(
            messages, tokenize=False, add_generation_prompt=True
        )
    else:
        text = _manual_llama3_prompt(messages)
    return tokenizer(text, return_tensors="pt", add_special_tokens=False).to(model.device)


# ------------------------------------------------------------------------- generate

GEN_KWARGS = dict(
    max_new_tokens=180,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
    repetition_penalty=1.1,
)


def reply(messages):
    inputs = build_inputs(messages)
    prompt_len = inputs["input_ids"].shape[-1]

    with torch.inference_mode():
        out = model.generate(
            **inputs,
            eos_token_id=terminators,
            pad_token_id=tokenizer.pad_token_id,
            **GEN_KWARGS,
        )

    # Only decode what was newly generated.
    new_ids = out[0][prompt_len:]
    text = tokenizer.decode(new_ids, skip_special_tokens=True).strip()

    # Belt-and-braces: cut anything after a leaked role header.
    for marker in ("user\n", "User:", "assistant\n", "Assistant:", "<|"):
        if marker in text:
            text = text.split(marker)[0].strip()
    return text


# ----------------------------------------------------------------------- chat loop

def chat():
    history = [{"role": "system", "content": SYSTEM_PROMPT}]
    print("Rufus dey online. Type 'exit' to comot, 'reset' to clear gist.\n")

    while True:
        try:
            user_msg = input("You: ").strip()
        except (EOFError, KeyboardInterrupt):
            print("\nLater!")
            break

        if not user_msg:
            continue
        if user_msg.lower() in {"exit", "quit"}:
            print("Later!")
            break
        if user_msg.lower() == "reset":
            history = [{"role": "system", "content": SYSTEM_PROMPT}]
            print("(gist cleared)\n")
            continue

        history.append({"role": "user", "content": user_msg})
        answer = reply(history)
        history.append({"role": "assistant", "content": answer})
        print(f"Rufus: {answer}\n")

        # Keep system prompt + last 8 turns so context no go blow up.
        if len(history) > 17:
            history = [history[0]] + history[-16:]


if __name__ == "__main__":
    chat()

Limitations

• No dataset card, training script, or evaluation results are published alongside this checkpoint, so its Pidgin-generation quality, coverage of dialectal variation, and safety behavior have not been independently verified.
• Inherits the general limitations of the Llama 3.2 3B base model, including potential factual errors, hallucinations, and biases.
• Only an F16 GGUF export is provided; no quantized variants (e.g. Q4/Q8) are currently included in this repository.
• License terms follow Meta's Llama 3.2 Community License, which applies to derivative/fine-tuned models.

Author

Developed by Ephraimmm.