Open research · RRRTX Labs

Foundation-model research, built to be inspected.

XRFM pairs an original decoder-only model trained from random initialization with a separately attributed defensive-coding track. Architecture, data, lineage and limitations are documented—not implied.

25.2M / 86.4M native checkpoints 20.08M prepared tokens 32,768 T4 vocabulary 3.10–3.12 CI matrix
Dual-track design

Two lineages. No blended claims.

Native and CoderSec share evaluation tooling and repository practices. They do not share weights, initialization or training history.

Track A · Native

Original transformer, from zero

25M and 85M parameter checkpoints trained from random initialization on a 20.08M-token multilingual and code corpus.

  • Grouped-query attention, RoPE, RMSNorm, SwiGLU
  • Exact-resume probe with zero weight difference
  • Per-domain held-out BPB measurements
View Native 25M →
Track B · CoderSec

Attributed QLoRA security track

Built on Qwen3-1.7B with NF4 + FP16 QLoRA. The base model, revision and Apache-2.0 license are stated explicitly.

  • Base revision 70d244cc
  • SFT dataset revision 808d7505
  • Checkpoint publication: not available
View CoderSec status →
Research artifacts

The ideas behind the metrics

Original rendered artifacts used throughout the site: the transformer stack, the defensive-security boundary and the multilingual token lattice.

Layered glass neural network sculpture with glowing cyan connections
Architecture

Decoder stack

Layered attention blocks with cached RoPE and gated feed-forward networks.

Read the architecture →
Measured, not marketed

What the published checkpoints show

BPB is bits per byte of held-out text. Lower is better. It measures likelihood fit—not coding skill, factuality or security capability.

ModelTraining evidencePer-domain BPBInterpretation
Native 25M900 steps · 58.98M processed tokensAll eight domains improved from random-init baselineArchitecture and tokenizer research result
Native 85M160 steps · 10.49M processed tokensAll eight domains improved, but remain above 25M final BPBShort scaling probe, not a stronger assistant
CoderSecQLoRA experiments on Kaggle T4Limited internal red-team suite onlyNo public checkpoint; no production claim

Read the full evaluation methodology →

Reproducibility

Pinned artefacts

Model, dataset and tokenizer revisions are recorded in model cards and release reports.

Research records →
Engineering

Continuously verified

CI validates 3.10, 3.11 and 3.12, packaging, type checks, both formatters, security and a CPU training smoke test.

Actions history →
Open contribution

Clear boundaries

Research implementations preserved; licensing is artifact-specific; known data defects are documented.

Contributing →