Model lineage
XRFM Native 25M was initialized randomly and trained on xrfm-native-shards-v1. No third-party pretrained weights were loaded. The checkpoint revision documented here is 3487b9c8.
Architecture
- Decoder-only transformer,
XRFMModel - Grouped-query attention: 6 query heads, 2 KV heads
- RoPE, RMSNorm, SwiGLU
d_model=384, 8 layers,d_ff=1024, context 2,048- 25,200,000 tied parameters
Training
900 steps over 58,982,400 processed tokens on Tesla T4 hardware. The dataset contains 20,080,975 unique prepared tokens; the larger figure reflects repeated passes. Wall time was 1,572.03 s with peak VRAM 3,865.93 MB. Exact-resume testing produced a maximum weight difference of 0.0.
Tokenizer and languages
The T4 Script-Protected Byte-Level BPE tokenizer has a 32,768 vocabulary and was selected in a four-way tournament. Evaluation domains are English, code, Urdu, Roman Urdu, Arabic, Hindi, Spanish and French.
Intended use and limits
Use for architecture research, tokenizer studies and reproducibility experiments. Out of scope: production chat, coding agents, security advice and high-stakes decisions. The model is not instruction-tuned and can generate incorrect or biased text.