Benchmarks

Published benchmark families, governance controls, and experiment guidance.

SynthWorld benchmark families pair deterministic public inputs with separately typed evaluator truth. Publication status is an explicit governance decision; the presence of a generator or schema does not by itself make a benchmark published.

Start here

• Registry catalogue is the generated public projection of capability and benchmark status from the resolved governance registries.
• Benchmark reference lists the stable catalog surface and explains how publication status is represented.
• Benchmark model defines the package’s benchmark boundary and independent metrics.
• Benchmark publication explains the gates between candidate, reviewed, and published artifacts.
• Public versus evaluator artifacts describes the physical and typed split that protects product-facing inputs from answer keys.
• Experiments records experiment-oriented guidance without presenting provisional results as product claims.
• Enterprise Identity Planning separates architecture-planning workflows from governed benchmark evaluation and marks current versus planned composition explicitly.

Existing frozen 1.0.0 benchmark artifacts remain byte-identical. Follow the publication status in the benchmark reference rather than inferring support from repository paths or examples.