NULLXES-LÆTEX-20B-Genesis-v0.1

Weight Genesis scaffold of the NULLXES-LÆTEX Stage2 dense proxy (~18.8B params, A35B-compatible width).

Developed by NULLXES · nullxesdai.online · Hub: MagistrTheOne

Status: research brick / architectural checkpoint.
Not a chat model. Random muP-init weights — expect nonsense generations until Stage2 train.
Next rework / first trained weights target: August 2026.

What this is

  • Own architecture: LatexForCausalLM (model_type=latex), NHAT hybrid attention
  • Shape: L=24, d_model=8192, GQA 64/8, d_ff=22016 → ~18.757B
  • Tokenizer: NULLXES-L-TEX-Tokenizer-v0.2 (131072 Unigram, full fill)
  • Init: muP + DeepNorm residual scaling, bf16 sharded safetensors
  • Intended path: Stage2 pretrain → A35B depth expand → future 480B-A35B MoE

What this is NOT

  • Not instruct / not SFT / not a Digital Employee personality
  • Not trained language competence (identity QA will fail on purpose)
  • Not distilled from Qwen / Llama / Mistral / DeepSeek / GLM
  • Not a replacement for 100M Stage0a (that one is identity-trained)

Load

import torch
from transformers import AutoModelForCausalLM, AutoConfig

repo = "MagistrTheOne/NULLXES-L-TEX-20B-Genesis-v0.1"
config = AutoConfig.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map="auto"
)

Roadmap

When Milestone
now Genesis v0.1 (this repo)
next week Stage2 ZeRO-3 smoke + ~100M tokens mid-eval on RTX PRO 6000
Aug 2026 First trained 20B weights / card refresh
later A35B dense → 480B-A35B MoE (cluster)

Contact

NULLXES · @MagistrTheOne · ceo@nullxes.com

Downloads last month
10
Safetensors
Model size
19B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support