feat: auto key-fix for troublesome LoRAs (alpha injection + key filtering)

#7
by LPX55 - opened
Owner

LoRA Auto Key-Fix (PR #7)

Adds automatic repair for "troublesome" LoRAs whose safetensors omit the <module>.alpha keys diffusers' Qwen LoRA converter expects (raising KeyError: 'img_in.alpha'), or include ComfyUI delta/patch keys (.diff, .diff_b) that diffusers' strict converter rejects (ValueError: state_dict should be empty...).

What changed

Three new helper functions (lines ~400โ€“500):

  1. _inject_missing_alpha_keys(state_dict) โ€” injects <module>.alpha = rank for every LoRA module, in BOTH diffusion_model.<module>.alpha AND stripped <module>.alpha forms (diffusers may strip the prefix before lookup). Fixes KeyError: 'img_in.alpha'.

  2. _filter_to_diffusers_lora_keys(state_dict) โ€” keeps only keys diffusers can consume (*.lora_up.weight, *.lora_down.weight, *.lora_mid.weight, *.alpha + normalizes *.lora_alpha โ†’ *.alpha), drops ComfyUI delta keys (*.diff, *.diff_b) and any other noise. Returns cleaned dict + stats.

  3. _duplicate_stripped_prefix_keys(state_dict) โ€” duplicates any diffusion_model. prefixed LoRA keys in unprefixed form, ensuring compatibility regardless of whether diffusers strips the prefix.

  4. _load_lora_weights_autofix(path, adapter_name) โ€” wraps pipe.load_lora_weights. Tries normal load first; on KeyError or ValueError, downloads the file, runs the three helpers, loads the cleaned dict. Runs automatically for every custom LoRA (no opt-in flag needed).

Modified load_custom_lora

  • Replaces direct pipe.load_lora_weights(path, ...) call with _load_lora_weights_autofix(path, CUSTOM_LORA_ADAPTER).
  • Adds safetensors.torch.load_file import (was missing) and traceback import.

Why automatic is better than opt-in

The reference space (cruisewagner2220) uses a needs_alpha_fix: True per-LoRA config flag. Ours detects the failure mode at load time and self-repairs โ€” covers alpha-missing AND leftover-keys errors without per-LoRA configuration.

LPX55 changed pull request status to merged

Sign up or log in to comment