Run the conformance suite#

This guide runs the portable OSDL conformance suite against an implementation's output. Conformance is how an engine substantiates the claim that it implements OSDL Core or a component library.

What a suite contains#

A conformance suite is a manifest plus cases. Each case names a model document, the experiment to run, and the artifacts to compare. The specification repository ships the core suite under conformance/core/, and each standard library carries its own suite that runs after core.

The comparison targets are recorded results and portable events, produced as files:

Produce and compare#

Run each case's document through your implementation and write the artifacts into one directory, using the exact file names above. Then, from a checkout of github.com/simcrafthq/osdl:

npm run conformance -- conformance/core/manifest.json path/to/actual

The runner resolves the suite extension graph, runs parent suites before the selected one, and reports each case's comparison. Run it in a trusted workspace: it rejects paths that escape the manifest directory, but assumes nothing else is rewriting files while it runs.

Comparison rules#

Result comparison is structural. Object member order is ignored, array and time-series point order matter, and numeric comparison follows the case's declared comparison.floatMode:

• bits, the default, requires equal IEEE 754 binary64 values and distinguishes the two zero signs.
• number uses numeric equality. A child suite can strengthen number to bits, and can never weaken bits.

Portable event files omit per-stream seq and wallTime fields, and the comparison allows independent events at the same simulation time to appear in a different order while preserving order within each causal identity, such as one entity.

Declaring support#

Support is declared per library namespace and version, outside any OSDL document. An implementation that claims [email protected] runs the core suite and then the des suite. Claiming one library never requires another.

Read more#

• Conformance in the specification: roles, suites, and extension rules.
• The conformance manifest schema.
• The reference machine: where the expected artifacts come from.