Technical architecture

How XRFM is built

The canonical implementation lives in src/xrfm/. Native models use the original XRFMModel; CoderSec uses an attributed pretrained base and project-owned fine-tuning tooling.

Three-dimensional layered neural network artifact
System view

From shards to checkpoints

The native pipeline is a research instrument: every stage is inspectable and reproducible.

XRFM native training and inference architecture Text / code shards uint16 · seq 2048 + label T4 byte-level BPE 32,768 tokens XRFMModel random initialization Trainer loss · checkpoint · resume safetensors HF release Decoder block × N RMSNorm GQA + RoPEcausal mask · KV cache RMSNorm SwiGLU MLPgated feed-forward Inference path GenerationEngine → sampling → decode Evaluation path per-domain BPB from held-out shards

Native pipeline. CoderSec reuses the data/evaluation tooling but replaces the model block with Qwen3-1.7B + QLoRA; no weights are combined.

Core components

Attention and normalization

Native models use grouped-query attention so query heads outnumber key/value heads, reducing cache size. Rotary position embeddings are cached with FP32-safe computation; RMSNorm stabilizes residual streams; SwiGLU provides a gated feed-forward network.

Tokenizer contract

T4 is a byte-level BPE with atomic special tokens and script-protection behaviour. The same tokenizer file is packaged with the library and published with both native checkpoints. Historical `vocab.json` remains the legacy control for the tokenizer tournament.

Checkpointing

Training writes serialized checkpoints and telemetry. The documented CUDA exact-resume probe compared weights after uninterrupted and resumed runs, reporting zero maximum absolute difference.

Dual-track boundary

Track A owns the transformer implementation and T4 tokenizer. Track B owns QLoRA dataset handling, adapter training, merging and defensive-security tooling. Shared scripts handle evaluation and reproducibility, but no Native weights are loaded into CoderSec and no Qwen weights are loaded into Native models.

The packaging boundary is enforced by pyproject.toml: only the xrfm package is shipped in the wheel; the API service and scripts remain repository tools.

Research preservation

Experimental NeuroTopo modules and tokenizer tournament code are preserved in the package and excluded from core lint gates where appropriate. They are not part of the published Native or CoderSec checkpoints.