Core components
Attention and normalization
Native models use grouped-query attention so query heads outnumber key/value heads, reducing cache size. Rotary position embeddings are cached with FP32-safe computation; RMSNorm stabilizes residual streams; SwiGLU provides a gated feed-forward network.
Tokenizer contract
T4 is a byte-level BPE with atomic special tokens and script-protection behaviour. The same tokenizer file is packaged with the library and published with both native checkpoints. Historical `vocab.json` remains the legacy control for the tokenizer tournament.
Checkpointing
Training writes serialized checkpoints and telemetry. The documented CUDA exact-resume probe compared weights after uninterrupted and resumed runs, reporting zero maximum absolute difference.