Conformance Fixtures

Status: DRAFT v0.1

Language-neutral test fixtures for the contracts in wire-contracts.md and the behaviors in core-concepts.md. Every Benzene implementation runs the same fixtures through its own runner; an implementation that passes them (plus the live-interop tests described in porting-guide.md §3) is conformant. API shape is explicitly not part of conformance.

The .NET runner lives at test/Benzene.Conformance.Test/ and is the reference for how a runner consumes these files.

Fixture files

Which fixtures a given conformance claim requires (cloud-service-profile.md §5):

At Core level the mesh fixtures apply only to ports that implement the optional mesh module (mesh.md §7); a port without mesh skips them and remains Core-conformant.

Canonical handlers

Envelope cases run against a fixed set of handlers that every runner MUST register natively, with exactly these topics and behaviors:

No handler is registered for any other topic — cases targeting unregistered or empty topics verify the router's not-found / validation-error behavior.

Status mapping case format

http-status-mapping.json and grpc-status-mapping.json each list forward/reverse { "from": ..., "to": ... } rows. A forward row's from may be the sentinel "<unknown>" (a status outside wire-contracts §3's vocabulary) rather than a real status string; such a row also carries isSuccessful (boolean), since wire-contracts §4's per-protocol tables route an unknown status to the generic-success or generic-error row depending on it (§1.2's authoritative signal — see §3's note on unknown-status routing). Each fixture accordingly lists two "<unknown>" rows, one per isSuccessful value. isSuccessful is meaningless on any other row (a known status maps by its own row regardless) and MUST be omitted there.

Envelope case format

{
  "name": "unique-case-name",
  "request":  { "topic": "...", "headers": { }, "body": "..." },
  "expected": {
    "statusCode": "ok",
    "body": { "greeting": "Hello world" },
    "headers": { "content-type": "application/json" }
  }
}

• request is the inbound envelope (wire-contracts §1.1), verbatim.
• expected.statusCode is compared exactly.
• expected.isSuccessful, when present, is compared exactly against the response envelope's isSuccessful field (wire-contracts §1.2) — the wire's authoritative success/failure signal.
• expected.body, when present, is parsed JSON compared by subset: every field in the expected object must be present in the actual (parsed) response body with a deeply-equal value; extra fields in the actual body (including null-valued ones) are ignored. This is deliberate — implementations may enrich responses, and writers may emit or omit null properties (wire-contracts §6).
• expected.headers, when present, is compared by subset the same way (keys case-insensitive).
• expected.bodyExclude, when present, lists members that must not appear in the parsed response body at all — a negative assertion, mirroring transport-metadata-cases.json's headersExclude. A failure case listing "status" pins that no numeric status member (wire-contracts §1.3) leaks onto a transport with no HTTP response, e.g. the raw envelope; a member absent from expected.body is otherwise unconstrained (subset matching alone would permit it either way), so bodyExclude is the only way to pin genuine absence.
• Human-readable message wording (e.g. the detail/title text of router-generated or registry problem documents) is intentionally not asserted.

Problem details case format

problem-details-cases.json (wire-contracts §1.3, §3.1, §4.1) has three independent groups:

• registry — rows lists every problem-type registry entry (benzeneStatus/type/httpStatus) directly, with no message to build — the cheapest check, the same rationale as defaultMetadataKeys. unknownStatus pins the fallback for an application-defined status: no registry row, HTTP status 500.
• envelopeCases — cases in exactly the envelope case format above, run against the canonical conformance:problem handler (see Canonical handlers). They pin structured errors round-tripping through the problem document, application-defined type passthrough via appType, and (via bodyExclude) that framework-produced problem documents carry no instance and no numeric status off an HTTP transport.
• httpRules — required only for ports that ship an HTTP binding (see the fixture-claims table above; the same conditional shape as transport-metadata-cases.json). failureCases lists, per Benzene status, the httpStatus an HTTP-bound failure response for that status MUST carry both as its response line and as the problem document's status member — the two MUST come from the same mapping (wire-contracts §4.1) so they can never disagree. The response's content-type MUST be application/problem+json for a JSON-negotiated failure response. successCase pins that success responses are unaffected: no problem document, ordinary content type. Title/detail wording is never asserted, per the envelope case format rule above.

Transport metadata case format

transport-metadata-cases.json covers the transports that carry Benzene metadata natively — SQS/SNS message attributes, Pub/Sub attributes, Service Bus/Event Hub application properties, Kafka/RabbitMQ headers. Each exposes the same shape under a different native name (a string→string dictionary), so the cases are written against that neutral dictionary and each runner adapts a case to its own native message before decoding it.

{
  "name": "topic-resolves-from-the-reserved-key",
  "metadata": { "topic": "conformance:greet", "x-correlation-id": "abc-123" },
  "expected": {
    "topic": "conformance:greet",
    "headers": { "x-correlation-id": "abc-123" },
    "headersExclude": ["topic"]
  }
}

• metadata is the native metadata dictionary, before decoding.
• expected.topic is compared exactly; "" means the binding resolved no topic (not an error — what an empty topic means is the router's business, pinned by envelope-cases.json).
• expected.headers is subset-matched, keys case-insensitive, as for envelope cases.
• expected.headersExclude lists keys that must not appear as headers — a metadata key the binding consumed as routing information must not also leak into the header dictionary.
• expected.version, where present, is the resolved payload version.
• A case with "requires": "versioning" applies only to ports implementing payload versioning (benzene-version is tier C).

defaultMetadataKeys names the keys themselves, and topicSources records where each binding gets its topic. Both are directly assertable: a port can compare its own constant against defaultMetadataKeys.topic without building a message at all, which is the cheapest possible check and the one that catches a rename.

The names are configurable, so the fixture asserts two things. metadataCases run against the defaults — the baseline that lets two untouched Benzene services interoperate. overrideCases each carry a metadataKeys object the runner applies before decoding, and assert that the replacement actually routes and that the default key becomes an ordinary header once overridden. A port that hard-codes a name passes the first group and fails the second.

Bindings whose source is not metadata are listed deliberately: EventBridge routes on detail-type and DynamoDB Streams derives {tableName}:{eventName}, so those bindings MUST NOT require a topic attribute, and the metadata cases do not apply to them.

Why this fixture exists. The metadata key names are the one part of the wire contract that no other fixture touches — envelope, status and protocol-mapping cases never look at native metadata. A port can therefore rename or misspell the topic attribute, pass every other fixture, and still be unable to exchange a single queue message with another port. That is not hypothetical: it is exactly how the .NET and Python ports diverged when the key was briefly benzene-topic (work/benzene-naming-principle.md §3c, since reversed).

The EventBridge embedded-headers key (_benzeneHeaders, wire-contracts §2, tier D) is deliberately not pinned here: it is scheduled to be renamed to benzene-headers (work/benzene-headers-plan.md), and a fixture asserting the current spelling would have to change with it.

Mesh fixture formats

Subset matching is as for envelope cases, with one addition needed by these fixtures: arrays compare by exact length with per-element subset matching, and an expected empty array ([]) matches an actual array that is empty or absent (writers may omit empty collections).

• mesh-descriptor-cases.json — the runner registers the two canonical envelope handlers (above) natively, builds the service descriptor with the fixture's serviceInfo, and subset-compares expectedDescriptor. runtime and the hash value are per-port and not pinned; instead hash asserts the hash's properties: its sha256: + 64-hex format, invariance to instanceId, and sensitivity to serviceVersion and to the topic set.

• mesh-trace-cases.json — traceparent rows assert the join/reject rules of mesh §3 observationally (a valid header's ids are adopted; an invalid one yields a fresh 32-hex trace id and no parent). invocations rows run one envelope each through a pipeline with the trace middleware installed and the canonical handlers plus one extra canonical mesh handler registered:

  Topic	Request body	Behavior
  conformance:panic	any	Panics/throws unconditionally — pins the rule that a handler panic is traced as service-unavailable, not lost

  expectedEvent is subset-matched against the single TraceEvent exported for that invocation. (conformance:panic is registered only for these trace cases, not for descriptor or envelope cases.)

• mesh-collector-cases.json — each case's steps run in order against one fresh collector; each step is an envelope request/expected pair asserted like an envelope case. The benzene:mesh:query:* responses are asserted as the observable surface for the ingest/derivation rules of mesh §4–6; those query shapes are not themselves promoted contracts.

Contract Document and contract hash case formats

Both files pin contract-document.md. Every case group in contract-document-cases.json shares a documents map at the top of the file, keyed by id; each case references one document by documentRef rather than repeating it. expectedTopics and expectedComponents are compared as sets — order-independent, no duplicates expected either side — unlike the positional array comparisons used elsewhere in this directory, since a topic scope or a schema closure is a membership question, not a sequence.

• parseCases — each gives a documentRef (and, for the fail-loud case, options) and an expected (subset-matched: openapi, and per-entry requests[]/events[] rows keyed by topic, asserting versionPresent/version/reserved as applicable) or an expectedError (asserting the failure's unknownTopics and validTopics sets, contract-document.md §5.2). versionPresent: false asserts the field is genuinely absent, not an empty string (contract-document.md §2's absent-means-unversioned rule).
• topicScopeCases — each gives a documentRef, options (topics/includeReserved, both optional, contract-document.md §5.2), and the resulting expectedTopics set of surviving requests[] topic ids.
• schemaClosureCases — each gives a documentRef, a topic, and the expectedComponents set: every components.schemas key reachable from that topic's request and response by the walk of contract-document.md §5.3. Includes a $ref cycle case and an allOf/oneOf reach case, per that section's normative walk.

contract-hash-cases.json lists cases, each an input document (already in whatever projection the case is about — a whole-service document, or one already topic-scoped per §5.3) and the exact expectedHash a conformant normalize → canonicalJSON (RFC 8785/JCS) → SHA-256 pipeline (contract-document.md §6.2) MUST produce. Two cases are deliberately not already normalized (one carries example/messageEndpoint/transports, one carries reserved entries) specifically so their expectedHash — identical to the minimal case's — proves an implementation performs the strip, not merely hashes its input verbatim.

Mapping table format

{
  "forward": [ { "from": "ok", "to": "200" } ],
  "reverse": [ { "from": "200", "to": "ok" } ]
}

Each row is asserted independently against the implementation's forward/reverse mapper. Rows with from of "<unknown>" assert the mapper's default for an unrecognized input.