PRML v0.1 — Pre-Registered ML Manifest Specification

Status: frozen 2026-05-22, not a finished standard. Published under the Community Specification License 1.0 by a single editor; no standards body has adopted it, and it carries no presumption of conformity with any regulation. Settled: §3 canonicalization has not changed since the freeze; three defect reports accepted 2026-08-23 added rejection requirements for invalid input and changed no canonical byte sequence and no digest of any valid manifest; the four reference implementations (Python, JavaScript, Go, Rust) agree byte-for-byte on 21 conformance vectors, all four reject the 16 negative-conformance cases that are not YAML-specific, and the two that parse YAML reject all 20; the media type application/vnd.prml+yaml was registered with IANA on 2026-09-03 — registration is not endorsement. Not settled: a manifest and its digest carry no evidence of time on their own — only a time-anchored record, a digest countersigned by an independent timestamp authority or entered in a public transparency log, can establish that a criteria object existed no later than the anchored time. Even then it does not establish that the evaluation ran afterwards: this specification does not bind a criteria record to the execution it describes. Execution linkage is out of scope for v0.1 and is the main open gap; §3 also has no formal grammar, so an independent implementation must work from the normative prose and the conformance vectors. Implementation status: all four reference implementations were written by the editor, and all public registry records to date originate with the editor. No independent implementation is known to the editor; independent interoperability therefore remains unproven. A manifest is interpreted under the rules identified by its version field, so valid v0.1 manifests continue to verify under v0.1 rules as later revisions are developed. The v0.2 RFC froze on 2026-05-22 and its formal comment window is closed; general feedback and defect reports remain welcome at GitHub discussions or [email protected].

60-second walkthrough — commit the criteria before a run, get a SHA-256 receipt, record later revisions on the amendment chain. Live demo at /demo.

Loading specification…

Related documents

v0.2 RFC — frozen 2026-05-22 (comment window closed)
Editor's contact page — how to reach the editor, latencies, refusals
For audit practices — a proposed workflow for including PRML in client deliverables
EU AI Act Article 12 — working pattern — the logging pattern, with code
ML reproducibility audit method — re-runnable / reproducible / auditable, defined
Pre-registration for ML — the clinical-trials discipline ported to ML
Article 12 readiness self-assessment — 10 questions, browser-only, one-page diagnostic
Registry board · Locks feed · Verify — public commit log
Positioning matrix — PRML vs in-toto, SLSA, Model Cards, HELM, ClinicalTrials.gov
Canonicalization portability findings — three cross-language gotchas from the Node.js second implementation
EU AI Act Article mapping — fields → Articles 12, 17, 18, 50, 72, 73
Compliance landing — for compliance leads and notified body assessors
arXiv preprint — 14-page LaTeX, CC BY 4.0 (cs.CR submission in flight)
v0.2 RFC roadmap — ten changes, five open questions, freeze 2026-05-22
Four reference implementations — Python, Node.js (falsify-js), Go, Rust. All pass 21/21 vectors byte-for-byte; CI verifies 4 × 21 = 84 byte-for-byte agreements per push, plus the full reject suite (RJ-001..RJ-020) against every implementation — 72 rejection checks in total, because RJ-015..RJ-018 are YAML-only and the JSON-reading implementations skip those four.
Conformance test vectors — 21 vectors total (13 v0.1 normative + 8 v0.2 candidate), all with locked SHA-256 digests.
GitHub Action: prml-verify — composite Action on the GitHub Marketplace; tags continuous-integration + code-quality; wraps the falsify CLI for one-line CI integration.
MLflow plugin: mlflow-falsify — pip install mlflow-falsify auto-tags every MLflow run with the PRML manifest hash + key fields. Zero workflow change. Source: studio-11-co/mlflow-falsify.
Cookbook: falsify-cookbook — field manual for the spec. 10 patterns + 4 anti-patterns; each one page with a runnable example. CC0.
Disclosure Index — public scorecard of how 27 well-known ML eval claims meet the 9 PRML disclosure criteria. Source: studio-11-co/falsify-integrity-index. CC0 data, MIT tooling.
Inspect AI adapter: falsify-inspect — anchor an Inspect AI eval claim's threshold to a SHA-256 hash before the run, verify the post-run log against it. MIT.