Running the conformance kit in CI

Thin recipe for implementers: how to run @ggui-ai/protocol-conformance in your own CI so a protocol regression on either side shows up as a red build. What conformance means is on the Conformance page; this page is only the wiring.

Install

The kit is a plain npm package with no monorepo assumptions — a plain npm install of the published tarball resolves the kit plus @ggui-ai/protocol and ws, and runs standalone (verified against @ggui-ai/protocol-reference-server installed the same way: 9 pass / 0 fail / 3 skip).

npm install -D @ggui-ai/protocol-conformance

Invoke

Programmatic runner (the real pass/fail signal). Boot your implementation, hand the runner a ConformanceHost so setup directives (session provisioning, fault injection) can run, and apply the kit’s red-build guard:

import { runConformance } from "@ggui-ai/protocol-conformance";

const result = await runConformance({
  serverUrl: "http://127.0.0.1:3100", // bare origin — the runner derives ws://…/ws
  auth: { kind: "bearer", token: process.env.TOKEN },
  host: myConformanceHost, // dispatchSetup / dispatchTeardown / readSessionField
});

// A failed fixture is a red build — and so is a run that executed
// ZERO fixtures (all skips prove nothing).
if (result.failed.length > 0 || result.passed.length === 0) process.exit(1);

CLI (wire-level smoke only). The bundled CLI has no host wiring, and every fixture in the current catalog declares setup steps — so a hostless CLI run skips them all and exits 2 by design. Use it as a connectivity/handshake smoke check, not as your conformance gate:

npx ggui-protocol-conformance --url ws://127.0.0.1:3100/ws --auth bearer:$TOKEN

Exit codes: 0 — at least one fixture executed and none failed; 1 — any fixture failed; 2 — invocation error or a zero-executed (all-skip) run. A run that proved nothing never reads as success.

What a failure means

• FAIL — your implementation violates a protocol obligation the fixture freezes (wrong ack shape, missing CONTRACT_VIOLATION rejection, version-handshake drift, …). Path-A fails are server-side vendor-neutrality bugs: fix the implementation, not the fixture.
• SKIP — not a failure. Path-B (browser-host) claims and setup directives your host doesn’t implement skip with a precise reason. But pin your pass and skip sets as exact sets in CI: a fixture silently degrading to a skip should fail your build, and re-pinning should be a deliberate act.
• All-skip — red (exit 2 / the programmatic guard above). Zero executed fixtures carries no signal.

A change is breaking iff a fixture that passed against version N fails against N+1 — the kit, not opinion, is the arbiter (see Version policy).

See also

• Conformance — the contract/protocol bars the kit makes observable.
• @ggui-ai/protocol-reference-server — a minimal target to sanity-check your CI wiring against before pointing the kit at your own implementation.
• Downstream SDK integrations — including the platform-composed (guuey-sdk) golden-path sample — gate their side of the seam with this same kit in their own CI; the protocol itself requires no particular platform, and the kit runs against any implementation of the wire.