Benchmark Terminology

KynML describes model training and evaluation. Kynetra Prime Benchmark Foundry measures the larger system around it. The two products share metric names and evidence fields, but they keep separate ownership boundaries.

Ownership Boundary

Surface	Owns
KynML	`.kyn` datasets, models, training recipes, built-in model metrics, generated run manifests
Prime Benchmark Foundry	The canonical 100-metric system catalog, normalized scoring, evidence coverage, layer summaries, and run comparison
Kynetra Foundry	Training recipe execution, evaluation gates, promotion decisions, rollback, and adapter lineage

KynML v1 does not accept all Prime system metrics inside an evaluate: block. The supported .kyn metrics remain mae, mse, rmse, and accuracy. System benchmarks travel beside a KynML run in a benchmark manifest.

Canonical Record

A portable benchmark observation uses these fields:

{
  "benchmarkId": "PRIME-BENCH-009",
  "term": "training_peak_memory",
  "value": 8192,
  "unit": "mib",
  "direction": "lower",
  "capturedAt": "2026-07-19T00:00:00.000Z",
  "sampleSize": 3,
  "evidenceState": "measured",
  "evidenceUri": "artifact://training/profile.json"
}

Core Vocabulary

Term	Meaning
`benchmark`	A versioned measurement definition with a target, critical boundary, unit, and direction
`observation`	One measured or derived numeric value for a benchmark
`benchmark_run`	A subject, version, timestamp, and set of observations scored together
`profile_id`	Stable workload/environment profile required for compatible comparison
`applicable_benchmark_ids`	Explicit benchmark pack in scope for one run; omitted means the full catalog
`baseline_run`	The prior compatible run used for comparison
`target`	The value at which a benchmark receives full normalized score
`critical`	The value at which a benchmark receives zero normalized score
`direction`	`higher` or `lower`; determines how raw values become normalized scores
`unit`	Stable measurement unit such as `percent`, `milliseconds`, `tokens`, or `mib`
`sample_size`	Number of cases represented by an observation
`evidence_state`	`measured`, `derived`, or `not_collected`
`evidence_uri`	Stable pointer to the profiler trace, eval report, ledger, or manifest
`coverage_percent`	Collected applicable benchmarks divided by all applicable benchmarks
`normalized_score`	Direction-aware score from 0 to 100
`regression`	A meaningful normalized score decrease against a compatible baseline
`promotion_gate`	Evidence rule that must pass before an adapter or model can be promoted
`lineage`	Source, transformation, recipe, model, hardware, and artifact history

Namespaces

Prime groups the 100 benchmarks into ten namespaces. KynML producers use the same snake-case terms when they emit compatible artifacts.

Namespace	Prime layer	Foundry stage	Examples
`training`	Training and Data	`curate`	`licensed_source_ratio`, `training_peak_memory`
`evaluate`	Learning and Evaluation	`evaluate`	`holdout_pass_rate`, `calibration_error`
`memory`	Memory and Knowledge	`remember`	`memory_leak_slope`, `cross_session_recall`
`context`	Retrieval and Context	`ground`	`long_context_accuracy`, `context_overflow_rate`
`tokens`	Token and Cost Efficiency	`optimize`	`tokens_per_success`, `quality_per_1k_tokens`
`runtime`	Inference and Runtime	`serve`	`time_to_first_token_p50`, `inference_peak_memory`
`tools`	Tools and Workflows	`execute`	`tool_selection_accuracy`, `idempotent_replay`
`swarm`	Agents and Swarms	`orchestrate`	`coordination_overhead`, `swarm_scaling_efficiency`
`safety`	Safety and Reliability	`gate`	`policy_compliance`, `rollback_success`
`scale`	Scale and Verified Impact	`promote`	`horizontal_scaling_efficiency`, `verified_outcome_rate`

Naming Rules

Metric terms use lower snake case: training_peak_memory.
Public benchmark slugs use kebab case: training-peak-memory.
Stable IDs use PRIME-BENCH-NNN and are never recycled.
Units are explicit and do not appear inside the term name.
Percent values use the 0-to-100 scale, not 0-to-1.
Latency percentile belongs in the name: memory_read_p95.
Resource scope belongs in the name: training_peak_memory versus inference_peak_memory.
Missing evidence is not_collected, never numeric zero.
Derived evidence is labeled derived and requires review before promotion.
Baseline comparison requires the same subject and profile_id.
Observations outside applicable_benchmark_ids are invalid, not silently ignored.

KynML Run Mapping

KynML artifact	Benchmark use
`.kyn` source	Recipe intent and configuration lineage
`run_manifest.json`	Seed, config hash, versions, dataset, and output lineage
Printed evaluation metrics	Model-quality observations for an applicable eval pack
Runtime profiler	Training throughput and peak-memory observations
Exported model	Artifact identity used in paired baseline/candidate evaluation

The canonical system registry is exported by @kynetra/prime-core/prime-benchmark-foundry, served at GET /api/v1/benchmarks, and evaluated at POST /api/v1/benchmarks. KynML documentation references that registry instead of maintaining a second copy of 100 definitions.