README.md

---
language:
  - uk
license: apache-2.0
tags:
  - text-generation
  - headline-generation
  - ukrainian
  - gpt2
  - transformers
  - peft
  - lora
pipeline_tag: text-generation
library_name: transformers
base_model:
  - benjamin/gpt2-large-wechsel-ukrainian
datasets:
  - FIdo-AI/ua-news
---

# NLPForUA/gpt2-large-uk-title-generation

> **📌 Educational release (Lab work)**  
> This model was trained and published as part of an educational lab assignment on **news title generation** using **GPT-2 + LoRA fine-tuning**.  
> The full training notebook (data prep → fine-tuning → evaluation) is available here:
>
> 
> [https://github.com/niksyromyatnikov/opnu-ml-assignments/blob/main/news-title-generation-gpt2-lora/news-title-generation-gpt2-lora.ipynb](https://github.com/niksyromyatnikov/opnu-ml-assignments/blob/main/news-title-generation-gpt2-lora/news-title-generation-gpt2-lora.ipynb)
>

**NLPForUA/gpt2-large-uk-title-generation** generates Ukrainian-style **news headlines** from article text.  
It is designed for experimenting with **prompting, decoding strategies, and parameter-efficient fine-tuning (LoRA)**.

---

## What this model does

Given a Ukrainian news article (or its first paragraph), the model generates a short candidate title/headline.

**Recommended prompt pattern:**
- article text  
- blank line  
- `Назва:`  
- generation

---

## Model details

- **Architecture:** GPT-2 Large (decoder-only causal LM)
- **Parameters:** ~774M (GPT-2 Large class)
- **Language:** Ukrainian (`uk`)
- **Task:** title / headline generation
- **Fine-tuning:** PEFT **LoRA** (parameter-efficient adaptation)

---

## Training data

The model was fine-tuned on Ukrainian news articles with associated headlines from:

- **Dataset:** `FIdo-AI/ua-news`

Typical categories include politics, economy, sports, tech, society, etc.

---

## Intended use

### ✅ Good for
- Headline suggestions for Ukrainian text
- Generating multiple candidate titles for editorial review
- Educational demos of LoRA fine-tuning and decoding
- Dataset prototyping / augmentation (with human filtering)

### ❌ Not intended for
- Fully automated publishing without a human editor
- High-stakes applications requiring guaranteed factual accuracy
- Legal/medical/safety-critical content generation

---

## Usage (Transformers)

```python
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

model_id = "NLPForUA/gpt2-large-uk-title-generation"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

articles = [
    "Уряд України розглядає нові зміни до податкового законодавства. Експерти прогнозують вплив на малий бізнес та ІТ-сектор..."
]

def predict_title(model, inputs: list, postprocess=False, temperature=0.6, max_new_tokens=48) -> list:
    outputs = []
    with torch.no_grad():
        for idx, row in enumerate(inputs):
            if (idx+1) % 100 == 0:
                print(f"Generated {idx+1} titles\n")
            prompt = row + "\n Назва:"
            batch = tokenizer([prompt], return_tensors='pt').to(device)
            output_tokens = model.generate(
                **batch,
                max_new_tokens=max_new_tokens,
                do_sample=True,
                temperature=temperature,
                pad_token_id=tokenizer.eos_token_id,
                bos_token_id=tokenizer.bos_token_id,
                eos_token_id=tokenizer.eos_token_id,
            )
            output = tokenizer.decode(output_tokens[0], skip_special_tokens=True)
            if postprocess:
                output = output.split("\n Назва:")[1]
            outputs.append(output.strip())
    return outputs

predicted_titles = predict_title(model, articles, postprocess=True)

for row in predicted_titles:
    print(row, '\n\n')
```