Train a Kynetra Prime.
Six stages. Five commands.
This is the runnable half of the Foundry: raw text in, a trained + adapted + evaluated
decoder out. Every stage carries the Foundry primitive it implements, ships a real
artifact, and has a gate before the next stage. Reference code:
kynetraecosystem/packages/forge — torch-only, ~1,100 lines.
The Spinefile — one file, both audiences
KynML `.kyn` · spec v2026.7Before the stages: the language update. A .kyn Spinefile is the whole run in ~30 prose-like lines a human reviews like a diff — and a deterministic grammar a model writes and edits without a parser in its head. Wrong keys fail at file:line, not at step 4,000. It compiles 1:1 onto the JSON config (verified: 27/27 keys identical), and one command runs the declared pipeline.
forge "prime-tiny"
data { corpus data/corpus script byte dedup on val 1% }
model { context 256 layers 4 heads 4 kv 2 width 256 } # Basalt Core
train { steps 2000 batch 32 x 4 lr 6e-4 -> 6e-5 warmup 100 } # Forgecurve
adapt { rank 8 alpha 16 targets q_proj v_proj } # RankWeave
pipeline data -> train -> eval
python -m forge check configs/prime-tiny.kyn # parse → JSON, no torch needed python -m forge run configs/prime-tiny.kyn # the whole spine, one command
Gate: forge check prints the JSON you expected. Full grammar: packages/forge/KYN-SPEC.md.
Curate & Script
Glyphsieve · Curriculord · Latticescript (P03)The model IS the dataset. Paragraph-level dedup drops what you'd otherwise memorize twice; Curriculord orders documents short→long; the tokenizer (byte built-in, BPE optional) turns text into uint16 memmaps.
mkdir -p data/corpus && cp /path/to/*.txt data/corpus/
python -m forge data --config configs/prime-tiny.json
# → data/tokens-tiny/{train.bin, val.bin, tokenizer.json, meta.json}
Gate: meta.json shows a dedup % and token counts you actually believe. Unrepresentative data here is silent quality loss everywhere after.
Pretrain — Basalt Core
Basalt Core (P08) · Forgecurve · AnchorpointThe exact P08 spec as ~250 lines of PyTorch: pre-norm RMSNorm, SwiGLU FFN, RoPE (Helix Anchor), GQA (Attentryx Grip), SDPA causal attention, weight tying, scaled residual init. The loop: AdamW(0.9, 0.95), cosine LR with warmup, grad accumulation, bf16 on CUDA, exact-resume checkpoints. It prints your Chinchilla budget (D≈20N) at startup — under-training becomes a decision, not an accident.
python -m forge train --config configs/prime-tiny.json # device=mps params=2.96M tokens/step=32,768 chinchilla-optimal≈59M tokens # step 500 | loss 1.8342 | lr 5.4e-04 | 41,000 tok/s # eval 500 | val_loss 1.7910 | ppl 5.99 ↳ new best — saved runs/prime-tiny/best.pt # interrupted? resume with exact state: python -m forge train --config configs/prime-tiny.json --resume runs/prime-tiny/last.pt
Gate: val loss plateaus and tokens-seen ≥ the printed Forgecurve budget.
Adapt — RankWeave → GraftFold
RankWeave · GraftFold (P01)Freeze the base, weave rank-r LoRA deltas into q_proj/v_proj (~0.1–1% trainable), train on the domain corpus, then fold the adapters back into the base weights — GraftFold's +0ms promise, literally W += B@A·scale.
python -m forge adapt --config configs/prime-tiny.json \
--ckpt runs/prime-tiny/best.pt --data-dir data/tokens-domain
# RankWeave: 8 sites · rank 8 · trainable 65,536 (2.21% of 2,962,432)
# → adapter.pt + merged.pt
Gate: adapted val loss ≤ base on the target corpus.
Prove — Proofgrid
Proofgrid (P09)The same sealed bench for every checkpoint: fixed seed, fixed val split, fixed probe prompts. Emits proofgrid.json — comparable across runs, machines and time. If you can't measure it, you can't ship it.
python -m forge eval --config configs/prime-tiny.json --ckpt runs/prime-tiny/best.pt
# { "val_loss": 1.62, "perplexity": 5.06, "params_m": 2.96, "device": "mps", … }
Gate: perplexity within budget; probe completions coherent. Then sample: python -m forge sample --ckpt … --prompt "…"
The Forgecurve table
D ≈ 20N doctrine (P08)| Config | Params | Token budget | Hardware |
|---|---|---|---|
| prime-tiny | ~3M | ~60M | M-series Mac — prove the whole spine today |
| prime-125m | ~100M | ~2B | one 24GB CUDA GPU (bf16 + compile) |
| prime-1b | ~1.2B | ~23B | doctrine config — Shardbastion multi-GPU (Roadmap) |
Honest roadmap, named not faked: PreferLoom (DPO alignment), Errorforge/Nanocrush (post-training quantization — see the Precision Ladder), Shardbastion (FSDP sharding). KynML's compiler will target this spine: one .kyn file → these five commands.