Datasets

Traceable data, honestly described

Both datasets are published on Hugging Face. Counts below come from manifest files, repository inventories and bounded data inspection—not from repository names or assumptions.

Track A · Pretraining

xrfm-native-shards-v1

Revision 1652affc. Eight train shards are present and accessible; the dataset viewer shows only the 623-row validation split because .npy token shards are not rendered as tables.

Train sequences9,577 × 2,049 uint16
Train tokens19,613,696
Validation tokens467,279
Total prepared tokens20,080,975
Source documents29,491
Validation docs623

Sources and licenses

FineWeb-Edu and FineWeb-2 (ODC-By-1.0), OpenCoder Stage-2 (MIT), AI4Bharat Sangraha (CC-BY-4.0). Aggregate metadata: ODC-By. Source revisions are pinned in manifest.json.

Processing

SHA-256 and MinHash-LSH 8×4 deduplication; contamination check against configs/eval/frozen_benchmarks_v1.json; SHA-256 split with seed 42. Eight language/code validation sets: en, ur, ur-Latn, ar, hi, es, fr, code.

Track B · SFT

xrfm-codersec-sft-40k

Revision 808d7505. The actual file contains 24,100 rows, not 40,000. It is preserved unchanged because documented QLoRA experiments used this revision.

defensive_security12,000
multilingual_qa6,000
rag_citation4,000
code2,100
Unique ChatML records12,107
Exact duplicate rows11,993

Schema

record_id, source, language, chatml, domain, license, license_tier, code_blocks, provenance.

Row licenses

Apache-2.0 (18,000), CC0-1.0 (4,000), MIT (2,100). Dataset-level metadata is other; no single license covers all rows.

Glass token shards with abstract multilingual glyphs
Tokenization

From text to pinned token shards

Native shards use the T4 32,768 vocabulary; CoderSec SFT rows carry source, license and provenance fields.

Measured quality issues

CoderSec SFT: known defects

All 24,100 rows use corrupted ChatML markers: licsystem, licuser, licassistant and irit instead of <|im_start|> and <|im_end|>. The three-role structure is present, but the serialization is not canonical ChatML.

All 2,100 code blocks are generic stubs of the form def func_<id>(x, y) returning result = x + y # placeholder, regardless of the requested task. They parse as Python but do not solve the prompts. CyberNative rows are heavily duplicated: 12,000 rows collapse to 7 unique responses.

A bounded pattern scan found no obvious emails, AWS keys, private-key headers or token-like strings. This is not a formal privacy audit. The full measurement is published as QUALITY_REPORT.json.

Reproducible correction

scripts/filter_codersec_sft.py canonicalizes ChatML, drops placeholder-only code rows and deduplicates. Applied to the published revision it produces 10,007 records. The corrected output has not been published under the original revision; any release will be separately versioned so training data provenance remains intact.

Usage example

from datasets import load_dataset

ds = load_dataset(
    "json",
    data_files="https://huggingface.co/datasets/ahmadrrrtx/xrfm-codersec-sft-40k/resolve/main/data/data.jsonl",
)
print(ds["train"][0])