# Barbet 1B Agent SFT TW Full Fine-Tune Playbook

## Objective

Fully fine-tune `OpenFormosa/barbet-1b-base` on `voidful/agent-sft` and iterate
against `voidful/claw-eval-zh --language tw` until additional full-FT waves no
longer improve the selected checkpoint. The final model should be pushed to
Hugging Face under `voidful` as a public, full-weight model.

## Base Model And Tokenizer

- Base model: `OpenFormosa/barbet-1b-base`
- Tokenizer: `OpenFormosa/PangolinTokenizer`
- Barbet checkpoint SHA observed: `22e930425a401b064c62f3c1cee27125f3db7683`
- Architecture: `BarbetForCausalLM`
- FSDP wrap class: `BarbetDecoderLayer`
- Parameters: 1,088,124,920
- Embedding shape: `[114944, 1536]`

`OpenFormosa/barbet-1b-base` does not include tokenizer files in the same repo.
The model README states that it uses `OpenFormosa/PangolinTokenizer`, whose
special tokens include `<|system|>`, `<|user|>`, `<|assistant|>`, and tool-use
markers. A custom Jinja chat template is used for OpenAI-style messages and
tool calls.

## Data

- Source dataset: `voidful/agent-sft`
- Prepared source from prior Gemma run:
  - Train: `/work/voidful2nlp/gemma-agent-sft/data/agent_sft/train.jsonl`
  - Valid: `/work/voidful2nlp/gemma-agent-sft/data/agent_sft/valid.jsonl`
- Counts:
  - Train: 242,472 rows
  - Valid: 2,048 rows
- Smoke slice:
  - Train: `/work/voidful2nlp/barbet-agent-sft/data/agent_sft_smoke/train.jsonl`
  - Valid: `/work/voidful2nlp/barbet-agent-sft/data/agent_sft_smoke/valid.jsonl`
  - Counts: 512 train, 64 valid

### Bounded-Context Filter

The raw dataset contains a small number of extremely long conversations. With
Axolotl's chat-template path, those rows are tokenized in full before they are
truncated to `sequence_len`, so they can consume most of a 4-hour `dev` window
without producing a checkpoint. On 2026-06-23, a 1-GPU `seq512` run on the raw
242,472-row train file reached only `2,000/242,472` tokenized prompts after
about 4.5 minutes and showed an ETA of several hours.

Character-length distribution measured from the raw train split:

- p50: 4,085 chars
- p75: 9,230 chars
- p90: 18,442 chars
- p95: 26,774 chars
- p99: 54,577 chars
- p99.5: 83,341 chars
- p99.9: 168,545 chars
- max: 976,773 chars

The filter keeps examples that have at least one trainable assistant turn, end
with an assistant turn, and stay below a bounded content-char threshold. This is
still full-weight fine-tuning; it only removes outlier conversations that are far
beyond the active training context length.

Filter script:

```text
/home/voidful2nlp/barbet-agent-sft/scripts/filter_agent_sft.py
```

Filtered datasets:

| Variant | Purpose | Train kept | Valid kept | Path |
| --- | --- | ---: | ---: | --- |
| cap20K-lastassistant | 1-GPU fast candidate, seq512 | 219,762 / 242,472 | 1,849 / 2,048 | `/work/voidful2nlp/barbet-agent-sft/data/agent_sft_cap20k_lastassistant` |
| cap40K-lastassistant | 4/8-GPU candidate, seq1024/2048 | 236,030 / 242,472 | 1,993 / 2,048 | `/work/voidful2nlp/barbet-agent-sft/data/agent_sft_cap40k_lastassistant` |
| cap8K-lastassistant | long-context retry, seq2048 | 164,740 / 242,472 | 1,393 / 2,048 | `/work/voidful2nlp/barbet-agent-sft/data/agent_sft_cap8k_lastassistant` |

Each filtered directory contains a `manifest.json` with exact counts and drop
reasons. The HF staging script copies those manifests into
`data_filter_manifests/`.

## Slurm Strategy

- Fast smoke uses `dev` with 2 GPUs to validate model load, FSDP wrapping,
  backward, and checkpoint save.
- Full waves should use as many GPUs as scheduling permits. Initial target is
  one H200 node with 8 GPUs.
- `8gpus` had a much later estimated start than `dev` for smoke, so smoke was
  submitted to `dev`.
- For `dev` jobs, the maximum walltime is 4 hours. Because preprocessing happens
  inside the training job, the bounded-context filtered data is used to ensure
  the GPU reaches actual full-FT steps before the walltime expires.
- Pending 4-GPU and 8-GPU jobs keep their queue positions while their config
  files point to cap40K filtered data.

2026-06-23 update: cap40K `seq2048` and cap20K `seq1024` both stalled in
Pangolin tokenization on long rows while holding GPU allocations. The practical
way to keep GPUs busy on the `dev` partition is to reuse the already-built
cap20K `seq512` prepared cache:

```text
/work/voidful2nlp/barbet-agent-sft/prepared/fullft_wave001d_cap20k_seq512
```

That cache lets multi-GPU jobs skip the slow tokenization stage and move
directly to full-parameter FSDP training. Higher-throughput 4/8-GPU retries use
`micro_batch_size: 4` with the same full-weight save path.

2026-06-23 16:01 CST update: `dev` is the active production partition for this
work. The cluster also exposes `8gpus`, `16gpus`, `32gpus`, `64gpus`, and
`256gpus`, but the observed fast-turnaround path is one H200 node on `dev`.
Current full-FT runs request `--gres=gpu:8` for one complete H200 node; eval
jobs request one GPU except judge/core runs, which request five GPUs. Slurm has
an effective per-user active/submit limit around ten jobs, so lower-priority
training comparisons were cancelled when checkpoint evals and the long-context
wave needed slots.

## Training Runs

| Run | Config | Status | Notes |
| --- | --- | --- | --- |
| smoke | `configs/fullft_smoke_barbet1b_agent_sft_seq512_steps3.yml` | completed, Slurm `139603` | 2 GPU full-FT smoke on 512-row data slice; exit `0:0`; wrote full `model.safetensors`, no adapter artifacts |
| wave001 | `configs/fullft_wave001_barbet1b_agent_sft_lr2e-6_seed411_seq2048_steps400_save100.yml` | cancelled, Slurm `139628` | cap40K data, 2e-6 LR, seq 2048, 8 GPU `dev`. Cancelled after startup because tokenization stayed at 0% while holding 8 GPUs; one row showed 157k tokens before truncation. |
| wave001a | `configs/fullft_wave001a_4g_barbet1b_agent_sft_lr2e-6_seed411_seq2048_steps400_save100.yml` | cancelled | Replaced by shorter `wave001b` to improve chance of checkpointing within `dev` limit |
| wave001b | `configs/fullft_wave001b_4g_barbet1b_agent_sft_lr2e-6_seed412_seq1024_steps100_save25.yml` | cancelled, Slurm `139642` | cap40K data, 2e-6 LR, seq 1024. Cancelled before useful work and replaced by cached cap20K seq512 configs. |
| wave001c | `configs/fullft_wave001c_1g_barbet1b_agent_sft_lr2e-6_seed413_seq512_steps100_save25.yml` | cancelled, Slurm `139644` then `139649` | Raw full data, seq512. First attempt oversubscribed preprocessing (`dataset_num_proc=64` on 16 CPU); second attempt used 16 workers but raw outliers produced multi-hour tokenization ETA. Replaced by cap20K `wave001d`. |
| wave001d | `configs/fullft_wave001d_1g_barbet1b_agent_sft_lr2e-6_seed414_seq512_steps100_save25_cap20k.yml` | completed, Slurm `139733` | cap20K data, 2e-6 LR, seq 512, save/eval every 25 steps, 1 GPU `dev`, 16 CPU, 350G. `139661` successfully wrote the prepared cache, then failed because the 1-GPU config still had FSDP enabled. The config was corrected to no-FSDP and `139733` completed with checkpoints 25/50/75/100. |
| wave001h | `configs/fullft_wave001h_2g_barbet1b_agent_sft_lr2e-6_seed417_seq512_steps100_save25_cap20k.yml` | completed, Slurm `139738` | cap20K data, 2e-6 LR, seq 512, save/eval every 25 steps, 2 GPU `dev`, 32 CPU, 500G. FSDP enabled, reuses the cap20K prepared cache, completed with checkpoints 25/50/75/100. FSDP checkpoints require inference cleanup before eval/upload. |
| wave001i | `configs/fullft_wave001i_8g_barbet1b_agent_sft_lr2e-6_seed418_seq1024_steps150_save50_cap20k.yml` | cancelled, Slurm `139773` | cap20K data, seq1024, micro batch 2. Cancelled because tokenization still stayed at 0% while holding 8 GPUs. |
| wave001j | `configs/fullft_wave001j_4g_barbet1b_agent_sft_lr2e-6_seed419_seq1024_steps100_save25_cap20k.yml` | cancelled before start, Slurm `139772` | cap20K seq1024 candidate replaced by cached seq512 candidate to avoid preprocessing stalls. |
| wave001k | `configs/fullft_wave001k_8g_barbet1b_agent_sft_lr2e-6_seed420_seq512_mb4_steps200_save50_cap20k_cached.yml` | cancelled, Slurm `139790` | cap20K data, seq512, micro batch 4, 2e-6 LR. Cancelled and replaced by a higher-LR 8-GPU run after low-LR candidates showed little useful behavioral shift. |
| wave001l | `configs/fullft_wave001l_4g_barbet1b_agent_sft_lr2e-6_seed421_seq512_mb4_steps150_save50_cap20k_cached.yml` | cancelled, Slurm `139791` | cap20K data, seq512, micro batch 4, 2e-6 LR. Cancelled and replaced by the higher-LR comparison queue. |
| wave002a | `configs/fullft_wave002a_8g_barbet1b_agent_sft_lr1e-5_seed430_seq512_mb4_steps300_save50_cap20k_cached.yml` | completed, Slurm `139805` | cap20K data, seq512, micro batch 4, 1e-5 LR, 8 GPU `dev`, uses prepared cache. Eval loss: initial `2.904`, step 50 `0.8636`, step 100 `0.6105`, step 150 `0.5702`, step 200 `0.5607`, step 250 `0.5591`, step 300 `0.5590`. Full-weight FSDP checkpoints 50/100/150/200/250/300 were cleaned for inference eval. |
| wave002b | `configs/fullft_wave002b_4g_barbet1b_agent_sft_lr5e-6_seed431_seq512_mb4_steps200_save50_cap20k_cached.yml` | cancelled, Slurm `139806` | cap20K data, seq512, micro batch 4, 5e-6 LR, 4 GPU `dev`. Cancelled to free the Slurm submit slot for wave002a checkpoint evals and the long-context wave003a run. |
| wave003a | `configs/fullft_wave003a_8g_barbet1b_agent_sft_lr3e-6_seed440_seq2048_mb1_steps200_save50_cap8k_from_w2a300.yml` | completed, Slurm `139866` | Starts from cleaned full-weight `wave002a` checkpoint-300, uses cap8K data, seq2048, micro batch 1, 3e-6 LR, 8 GPU `dev`, save/eval every 50. Prepared cache completed with 108,002 valid training sequences and 927 valid eval sequences after dropping rows over seq2048 or without trainable tokens. Eval loss: initial `0.9641`, step 50 `0.9176`, step 100 `0.9021`, step 150 `0.8990`, step 200 `0.8988`. Checkpoint-50 was cleaned to `/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003a_ckpt50` and sent to automated TW eval as Slurm `139944`; checkpoint-100 was cleaned to `/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003a_ckpt100` and sent as Slurm `139974`; checkpoint-150 was cleaned to `/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003a_ckpt150` and sent as Slurm `140011`; checkpoint-200 was cleaned to `/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003a_ckpt200` and sent as Slurm `140047`. |
| wave003b | `configs/fullft_wave003b_8g_barbet1b_agent_sft_lr2e-6_seed441_seq2048_mb4_steps150_save50_cap8k_from_w2a100.yml` | completed, Slurm `139922` | Starts from cleaned full-weight `wave002a` checkpoint-100, the first completed automated-only TW improver. Uses the same cap8K seq2048 prepared cache but increases micro batch from 1 to 4 to better use H200 memory and compute. Eval loss: initial `1.002`, step 50 `0.9775`, step 100 `0.9707`, step 150 `0.9707`. First train steps use about 71GB per H200 and reach roughly 130-210 tokens/sec/GPU, compared with about 19GB and 30-70 tokens/sec/GPU in wave003a. Checkpoint-50 was cleaned to `/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003b_ckpt50` and sent to automated TW eval as Slurm `139995`; checkpoint-100 was cleaned to `/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003b_ckpt100` and sent as Slurm `140022`; checkpoint-150 was cleaned to `/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003b_ckpt150` and sent as Slurm `140048`. Full FT only, no LoRA/QLoRA/adapter/quantized loading settings. |
| wave001e | `configs/fullft_wave001e_4g_barbet1b_agent_sft_lr2e-6_seed415_seq1024_steps100_save25_cap40k.yml` | config template | Standalone cap40K 4-GPU config mirroring the patched `wave001b` setup |
| wave001f | `configs/fullft_wave001f_8g_barbet1b_agent_sft_lr2e-6_seed416_seq2048_steps400_save100_cap40k.yml` | config template | Standalone cap40K 8-GPU config mirroring the patched `wave001` setup |

## Kernel Notes

The existing `gemma4-agent` env has Torch `2.8.0+cu128`, but does not currently
have `mamba_ssm` or `causal_conv1d`. A direct pip install attempt was stopped
because the resolver started pulling `torch 2.12/cu13`, which would risk
breaking the working environment. Until an isolated compatible install is
prepared, Barbet will use the model's PyTorch Mamba fallback. This is correct
but slower, so the initial full wave uses seq 2048 rather than seq 8192.

## Evaluation

Primary evaluation target:

```text
voidful/claw-eval-zh --language tw --suite all --core
```

Judge model for judge-needed suites:

```text
google/gemma-4-31B-it
```

Evaluation runner:

```text
/home/voidful2nlp/barbet-agent-sft/slurm/eval_claw_barbet.sbatch
```

Barbet requires a custom Transformers OpenAI-compatible server because the
tokenizer lives in `OpenFormosa/PangolinTokenizer`; the evaluation runner passes
`--server-script /home/voidful2nlp/barbet-agent-sft/scripts/serve_barbet_openai.py`.

FSDP-saved checkpoints may write `_fsdp*.py` and set:

```json
"AutoModelForCausalLM": "_fully_shard.FSDPBarbetForCausalLM"
```

Those are valid training artifacts but not clean inference checkpoints. Before
evaluation or HF upload, clean them with:

```text
/home/voidful2nlp/barbet-agent-sft/scripts/clean_barbet_checkpoint_for_inference.sh SOURCE_CHECKPOINT DEST_DIR
```

The script keeps full `model.safetensors`, removes FSDP helper files, restores
`modeling_barbet.BarbetForCausalLM` in `config.json`, and blocks adapter/LoRA
artifacts.

Automated-only TW quick eval results:

| Candidate | Slurm | Status at 2026-06-23 16:01 CST | Score Snapshot | Output |
| --- | --- | --- | --- | --- |
| base `OpenFormosa/barbet-1b-base` | `139763` | completed | `2.7 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/base_automated_tw` |
| wave001d checkpoint-25 | `139756` | completed | `2.7 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave001d_ckpt25_automated_tw` |
| wave001h checkpoint-25 clean | `139778` | completed | `2.7 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave001h_ckpt25_clean_automated_tw` |
| wave001d checkpoint-100 | `139795` | completed | `2.7 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave001d_ckpt100_automated_tw` |
| wave001h checkpoint-100 clean | `139796` | completed | `2.7 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave001h_ckpt100_clean_automated_tw` |
| wave002a checkpoint-50 clean | `139821` | completed | `2.755 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave002a_ckpt50_clean_automated_tw` |
| wave002a checkpoint-100 clean | `139825` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave002a_ckpt100_clean_automated_tw` |
| wave002a checkpoint-150 clean | `139831` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave002a_ckpt150_clean_automated_tw` |
| wave002a checkpoint-200 clean | `139840` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave002a_ckpt200_clean_automated_tw` |
| wave002a checkpoint-250 clean | `139860` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave002a_ckpt250_clean_automated_tw` |
| wave002a checkpoint-300 clean | `139857` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave002a_ckpt300_clean_automated_tw` |
| wave003a checkpoint-50 clean | `139944` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003a_ckpt50_clean_automated_tw` |
| wave003a checkpoint-100 clean | `139974` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003a_ckpt100_clean_automated_tw` |
| wave003b checkpoint-50 clean | `139995` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003b_ckpt50_clean_automated_tw` |
| wave003a checkpoint-150 clean | `140011` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003a_ckpt150_clean_automated_tw` |
| wave003b checkpoint-100 clean | `140022` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003b_ckpt100_clean_automated_tw` |
| wave003a checkpoint-200 clean | `140047` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003a_ckpt200_clean_automated_tw` |
| wave003b checkpoint-150 clean | `140048` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003b_ckpt150_clean_automated_tw` |
| wave004a OPD checkpoint-25 clean | `140989` | completed | `2.755 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave004a_opd_ckpt25_clean_automated_tw` |
| wave004a OPD checkpoint-50 clean | `141001` | completed | `2.755 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave004a_opd_ckpt50_clean_automated_tw` |
| wave004a OPD checkpoint-100 clean | `140742` | completed | `2.790 / 25` | `/work/voidful2nlp/barbet-agent-sft/eval/wave004a_opd_ckpt100_clean_automated_tw` |

Full/core TW judge evals:

| Candidate | Slurm | Status | Judge | Output |
| --- | --- | --- | --- | --- |
| wave002a checkpoint-100 clean | `139916` | completed: `1.267 / 20` | `google/gemma-4-31B-it` | `/work/voidful2nlp/barbet-agent-sft/eval/wave002a_ckpt100_clean_all_core_tw_judge31b` |
| wave003a checkpoint-100 clean | `140087` | completed: `1.267 / 20` | `google/gemma-4-31B-it` | `/work/voidful2nlp/barbet-agent-sft/eval/wave003a_ckpt100_clean_all_core_tw_judge31b` |

## Current Best

Final release candidate: `wave003a` checkpoint-100, cleaned for inference at:

```text
/work/voidful2nlp/barbet-agent-sft/outputs/clean_infer/wave003a_ckpt100
```

The selected checkpoint is a full-weight fine-tune, not LoRA, QLoRA, adapters,
or quantized loading. It starts from the best seq512 wave and adds a seq2048
continuation pass. It tied the best automated-only TW score (`2.790 / 25`) and
tied the full/core TW judge score (`1.267 / 20`) with `wave002a`
checkpoint-100.

No later candidate improved either benchmark target. Every completed candidate
from `wave002a` checkpoint-100 through `wave003a`/`wave003b` final checkpoints
and the first on-policy distillation continuation tops out at `2.790 / 25` on
automated-only TW quick eval. The judge/core rerun for `wave003a`
checkpoint-100 matched `wave002a` checkpoint-100 at `1.267 / 20`. Long-context
continuation, higher GPU-utilization continuation, and the 256-example OPD
probe therefore did not move the benchmark beyond the first `wave002a` gain, so
the run reached a clear no-improvement plateau on 2026-06-23.

## On-Policy Distillation Continuation

New objective after the full-FT plateau: continue with on-policy distillation
without assuming the teacher and student share a vocabulary.

This rules out token-level KL over logits. The continuation uses sequence-level
distillation instead:

1. Sample a next-assistant response from the current Barbet student on real
   `voidful/agent-sft` conversation prefixes.
2. Give a stronger teacher (`google/gemma-4-31B-it`) the conversation prefix,
   available tools, the student sampled response, and the original dataset
   reference answer.
3. Ask the teacher to return only the corrected next assistant message as JSON:
   either assistant text or structured `tool_calls`.
4. Train Barbet on the teacher message with the normal Barbet/Pangolin tokenizer
   and chat template. Teacher/student token IDs never interact.

Implementation:

```text
/home/voidful2nlp/barbet-agent-sft/scripts/generate_on_policy_distill.py
/home/voidful2nlp/barbet-agent-sft/slurm/generate_on_policy_distill.sbatch
```

First continuation wave:

| Run | Purpose | Base | Distill data | Config |
| --- | --- | --- | --- | --- |
| wave004a | 256-example fast on-policy distillation first wave | `wave003a` checkpoint-100 | `/work/voidful2nlp/barbet-agent-sft/data/on_policy_distill/wave004a/train.jsonl` | `configs/fullft_wave004a_opd_barbet1b_lr1e-6_seed450_seq2048_mb4_steps100.yml` |

The first wave was intentionally bounded so it could complete quickly and be
evaluated against `claw-eval-zh --language tw`. It did not beat the best
automated score. Checkpoint-25 and checkpoint-50 regressed to
`2.754761904762 / 25`; checkpoint-100 tied the prior best at
`2.790476190476 / 25`. Because it did not exceed the automated screening
threshold, no full/core judge rerun was launched for wave004a.

Submitted Slurm chain on 2026-06-23:

| Stage | Job | Notes |
| --- | ---: | --- |
| Generate OPD data | `140738` | `LIMIT=256`, `STUDENT_BATCH_SIZE=8`, `TEACHER_BATCH_SIZE=4`, `STUDENT_MAX_NEW_TOKENS=128`, `TEACHER_MAX_NEW_TOKENS=384`; invalid teacher JSON falls back to the original reference assistant message |
| Full-FT continuation | `140740` | dependency `afterok:140738` |
| Clean + automated TW eval | `140742` | dependency `afterok:140740`, evaluates checkpoint-100 |
| Clean + automated TW eval | `140989` | checkpoint-25, score `2.754761904762 / 25` |
| Clean + automated TW eval | `141001` | checkpoint-50, score `2.754761904762 / 25` |
| Clean + automated TW eval | `141028` | checkpoint-75, cancelled after checkpoint-25/50 showed regression so checkpoint-100 could run sooner |

Earlier generation attempts `140635` and `140671` were cancelled before
training because the first was too slow and the second exposed invalid raw JSON
teacher fallbacks during sample inspection. The generator now avoids training on
raw invalid JSON.

Final wave004a result:

```text
wave004a checkpoint-25 automated TW:  2.754761904762 / 25
wave004a checkpoint-50 automated TW:  2.754761904762 / 25
wave004a checkpoint-100 automated TW: 2.790476190476 / 25
prior best wave003a checkpoint-100:   2.790476190476 / 25
```

Conclusion: wave004a OPD tied but did not improve the automated TW best. The
published full-finetuned model should remain `wave003a` checkpoint-100.