anonboltzinterp/Boltz1-SAEs-L2-rec0

Boltz-1 SAEs — Pairformer trunk, recycle 0

TopK sparse autoencoders trained on Pairformer trunk activations from Boltz-1, taken at recycle iteration 0 — the trunk's first pass, before any recycled structural state is folded back in. Published for anonymous double-blind review; no authorship or affiliation is attached to this repository.

Architecture

SAE type	TopK, k = 256
Latent width	2048
Input width	384
Weight L2	3e-3
Preprocessing	training-set mean subtracted before encoding
Decoder	unit-normalised
Training steps	500,000
Seeds per layer	3
Base model	Boltz-1

Every run carries its full hyperparameter set in config.json, where rec is 0 for this repository.

Contents

75 runs: 25 layers x 3 seeds. The layers are the even indices 0-46, plus layer 47 — matching the recycle-1 repository, so the two can be compared layer for layer.

layer<L>/
  pairformer<L>_topk256_lat2048_demean_longtrain500000_l2_3e-3_alive_cross_seed.json
  pairformer<L>_topk256_lat2048_demean_longtrain500000_l2_3e-3_seed<S>/
      checkpoint_step_500000.pt
      config.json
      mean_vector.npy
      eval_step_500000.json
      stats.jsonl

<S> is 1, 2 or 3.

Loading

from huggingface_hub import hf_hub_download

repo = "anonboltzinterp/Boltz1-SAEs-L2-rec0"
run = "layer24/pairformer24_topk256_lat2048_demean_longtrain500000_l2_3e-3_seed1"
for name in ("config.json", "mean_vector.npy", "checkpoint_step_500000.pt"):
    hf_hub_download(repo_id=repo, filename=f"{run}/{name}", local_dir="sae")

Cache recycle-0 and recycle-1 checkpoints in separate directories: the run names are identical between the two repositories, so a shared cache will serve the wrong weights.

Companion repositories

• anonboltzinterp/Boltz1-SAEs-L2-rec1 — the same trunk layers at recycle 1
• anonboltzinterp/Boltz1-SAEs-L2-Diffusion — diffusion-module SAEs