Testing and Drift

Private web app APIs drift without notice. SiteKit makes drift visible before it silently poisons a pipeline.

Fixture Tests

Fixtures are redacted captures checked into the adapter repository.

sitekit test destination_portal --fixtures

Fixture tests prove code generation, schemas, transforms, and known-fact assertions still work without touching the live site.

Live Tests

sitekit test destination_portal --live

Live tests run cheap, reviewed operations against the real site:

tests:
  events.list:
    - expect: array
    - minCount: 1
    - allHave:
        workspaceId: ws_123

Browser-backed live tests can also capture screenshots and traces:

sitekit test expense_portal --live --trace

Known-Fact Assertions

Some facts are not inferable from traffic. They are adapter tests anyway.

knownFacts:
  - name: workspace_scope
    assertion: every result has workspace_id equal to input.workspaceId
  - name: write_method
    assertion: records.update uses POST and sends full editable payload
  - name: graphql_errors_raise
    assertion: response.errors fails the call even when status is 200

Drift Report

When a site changes, SiteKit produces a specific report:

destination_portal.events.list failed

Schema drift:
  events[0].location_id
    expected: number
    received: string

Auth:
  unchanged

Pagination:
  unchanged

Suggested repair:
  update LocationId schema to string | number
  keep public client field as number by coercing numeric strings

Browser-backed drift reports include selector and screenshot context:

expense_portal.expenses.createDraft failed

Selector drift:
  getByLabel("Expense type")
    expected: select or combobox
    found: no matching element

Drift Telemetry

Production validation failures are useful drift signals:

drift signal
  adapter: destination_portal@0.1.4
  operation: events.list
  source: pipeline
  failure: schema_validation

Those signals can prioritize repair and live probes. Fixture tests are cheap and should run often. Live probes should be budgeted by adapter usage, operation criticality, and recent failures.

The repair suggestion may be LLM-assisted. The drift detection itself is deterministic.

Freshness And Demotion

Verification is not permanent. A live probe that passed months ago is evidence about the past, not proof that a private web app still behaves the same way.

verification:
  status: verified
  verifiedAt: 2026-08-27T10:22:00Z
  freshness:
    liveProbeMaxAgeDays: 30
    demoteTo: raw

When freshness expires or enough drift signals accumulate, SiteKit should reduce effective exposure automatically. The raw operation remains useful for repair, but MCP tools and unattended pipeline methods should fail closed until evidence is refreshed.