Conformance Suite

Overview

@grantex/conformance is a black-box test suite that validates any Grantex protocol server implementation. It makes real HTTP requests against a live endpoint and checks that responses match the protocol specification.

Latest Production Results

63 passed, 2 skipped, 0 failed — 65 total tests. See the full breakdown.

Use it to:

Verify your self-hosted deployment after setup or upgrades
Validate custom implementations of the Grantex protocol
Run in CI/CD to catch regressions before they reach production
Certify compliance with the Grantex spec

Install

npm install -g @grantex/conformance

Or run directly with npx:

npx @grantex/conformance --base-url http://localhost:3001 --api-key YOUR_API_KEY

Configure

The suite requires two parameters:

Parameter	Description
`--base-url`	Base URL of the Grantex auth service
`--api-key`	Developer API key with permissions to manage agents, grants, and audit

Usage

Run all core suites

grantex-conformance --base-url http://localhost:3001 --api-key sk_test_xxx

Run a single suite

grantex-conformance --base-url http://localhost:3001 --api-key sk_test_xxx --suite health

Include optional extensions

grantex-conformance --base-url http://localhost:3001 --api-key sk_test_xxx \
  --include policies,webhooks,scim

JSON output (for CI pipelines)

grantex-conformance --base-url http://localhost:3001 --api-key sk_test_xxx --format json

Stop on first failure

grantex-conformance --base-url http://localhost:3001 --api-key sk_test_xxx --bail

CLI Reference

grantex-conformance --base-url URL --api-key KEY [options]

Options:
  --base-url <url>       Base URL of the Grantex auth service (required)
  --api-key <key>        API key for authentication (required)
  --suite <name>         Run a specific suite only
  --include <ext,...>    Include optional extensions (comma-separated)
  --format <format>      Output format: text or json (default: text)
  --bail                 Stop on first failure
  -h, --help             Display help

Core Suites

The suite ships with 40 tests across 10 core suites. These cover every MUST requirement in the specification.

Suite	Tests	Spec	What it validates
`health`	2	§3.3	Health endpoint returns 200; JWKS contains valid RS256 keys
`agents`	5	§10	Agent CRUD: create with agentId + DID, list, get, update, delete
`authorize`	4	§5.1–5.2	Authorization request creation, field validation, consent flow
`token`	3	§5.3	Code exchange returns grant token; rejects invalid and reused codes
`tokens`	4	§7.2–7.3	Token verify, revoke, post-revoke verify, garbage token handling
`grants`	4	§7.1	Grant listing, retrieval, revocation, status transitions
`delegation`	5	§9	Delegation with JWT claims, scope enforcement, depth limits, cascade revocation
`audit`	5	§8	Audit log creation, hash chain integrity, entry retrieval, hash computation
`security`	5	§14	Auth enforcement, JWKS algorithm checks, scope escalation prevention, audit immutability
`rate-limit-headers`	3	§14	Rate limit header presence and format, JWKS endpoint exemption

Optional Extensions

Enable these with --include to test optional protocol features:

Suite	Tests	What it validates
`policies`	5	Policy CRUD and enforcement
`webhooks`	3	Webhook registration and management
`scim`	6	SCIM 2.0 provisioning endpoints
`sso`	4	SSO configuration and OIDC flow
`anomalies`	3	Anomaly detection and acknowledgement
`compliance`	4	Compliance reporting and evidence export

Automation

Invoke the CLI from CI and consume its JSON output:

npx @grantex/conformance \
  --base-url http://localhost:3001 \
  --api-key "$GRANTEX_API_KEY" \
  --format json > conformance-results.json

The package also exports supported programmatic entry points:

import { runConformanceTests } from '@grantex/conformance/runner';
import { reportJson } from '@grantex/conformance/reporter';

const report = await runConformanceTests({
  baseUrl: 'http://localhost:3001',
  apiKey: process.env.GRANTEX_API_KEY!,
  format: 'json',
  bail: false,
});

console.log(reportJson(report));

The report contains suite results and a summary with total, passed, failed, skipped, and duration fields.

Server Requirements

Your server must meet these requirements to pass the conformance suite:

Implement all core Grantex protocol endpoints (/v1/agents, /v1/authorize, /v1/token, /v1/tokens/verify, /v1/tokens/revoke, /v1/grants, /v1/grants/delegate, /v1/audit/log, /v1/audit/entries)
Serve a JWKS at /.well-known/jwks.json with RS256 keys
Return proper HTTP status codes (201 for creation, 204 for deletion, 400 for validation errors, 401 for auth failures)
Support the authorization code flow (sandbox auto-code or /v1/consent/:id/approve)
Enforce delegation scope constraints and depth limits

CI/CD Integration

Add conformance tests to your deployment pipeline:

# GitHub Actions example
- name: Run conformance suite
  run: |
    npx @grantex/conformance \
      --base-url ${{ secrets.GRANTEX_URL }} \
      --api-key ${{ secrets.GRANTEX_KEY }} \
      --format json \
      --bail

Exit Codes

Code	Meaning
`0`	All tests passed
`1`	One or more tests failed
`2`	Configuration or runtime error

Requirements

Node.js 18+

​Overview

Latest Production Results

​Install

​Configure

​Usage

​Run all core suites

​Run a single suite

​Include optional extensions

​JSON output (for CI pipelines)

​Stop on first failure

​CLI Reference

​Core Suites

​Optional Extensions

​Automation

​Server Requirements

​CI/CD Integration

​Exit Codes

​Requirements