Skip to content

Workspace support tickets runbook

Migration, backfill, refresh degradation, observability, and rollback procedures for workspace support tickets.

The Tickets page is a workspace-scoped projection of user-originated GitHub bug reports. The support_tickets table is owned by DB_0; GitHub remains the status source of truth. Members with workspace_data:read can read the active workspace only. Superadmins use the separate selected- workspace server function.

  • Every workspace member with workspace_data:read sees all tickets for the active workspace; the first release does not limit rows to the reporter. Reporter email is retained for provenance but is not shown as a separate UI field.
  • Closed issues are shown as Resolved unless GitHub explicitly reports state_reason=not_planned, which remains Not resolved.
  • Only dashboard-generated bug_report issues are included; operator-created repository issues remain outside the first-release projection.
  • Ambiguous historical attribution has no repair UI in v1 and is reported for manual follow-up.
  1. Apply migrations/drizzle/0049_workspace_support_tickets.sql to DB_0 with pnpm db:migrate:remote -- --yes (or the approved deployment wrapper).

  2. Deploy the dashboard with the support-ticket capture path enabled. Capture is best effort: a GitHub issue still succeeds if the local projection write fails, and the failure is recorded in GlitchTip.

  3. Run the backfill in dry-run mode and review the ambiguous/unmatched report:

    Terminal window
    pnpm db:support-tickets:backfill -- --remote --json --dry-run
  4. Apply only after review and explicit confirmation:

    Terminal window
    pnpm db:support-tickets:backfill -- --remote --apply \
    --confirm "I understand this writes support tickets to D1" --json

The backfill is idempotent on (github_repo, github_issue_number). It considers only issues with the generated ## Reporter marker and ignores pull requests. It assigns a historical issue only when the workspace marker is valid or reporter-email membership resolves to one workspace. Ambiguous and unmatched issues are reported, never guessed, and there is no v1 repair UI. The JSON created_events collection is the machine-readable event report for the backfill; it is intentionally report-only so a rerun cannot create duplicate activity/audit entries. Forward captures and visible status refreshes use the dashboard emitAppEvent seam.

  • List queries are filtered by the server-authenticated entity_id, ordered newest first, and refresh only stale rows on the visible page.
  • resolved is github_state = closed and github_state_reason != not_planned.
  • A GitHub 404, rate limit, malformed response, or missing token leaves the cached row in place, marks it stale, and returns a degraded notice. The page remains usable.
  • Set SUPPORT_TICKETS_GITHUB_REFRESH_ENABLED=false to serve cached rows only during an incident. Re-enable it after the GitHub token/API issue is repaired.
  • SUPPORT_TICKETS_STATUS_STALE_MS controls the refresh threshold (default 15 minutes; bounded to 24 hours).

GlitchTip events use the support-tickets source for list failures, status refresh failures, status-event failures, and capture failures. Cloudflare logs/metrics should be checked for:

  • support.ticket.created and support.ticket.status.synced activity events;
  • operation=status-refresh, including stale, refreshed, and failed counts;
  • HTTP 403/503 rates for the Tickets server functions;
  • GitHub API 401/403/404/429 responses and list latency.

A rollout is healthy when new issue capture succeeds, the first page remains below the dashboard read latency target, refresh failures remain below the agreed 2% threshold, and no workspace can observe another workspace’s entity_id rows.

  1. Set SUPPORT_TICKETS_GITHUB_REFRESH_ENABLED=false and stop the backfill.

  2. Hide/remove the Tickets route/nav deployment if the page itself is unsafe.

  3. Revert the bug-report capture hook; leave GitHub issue creation unchanged.

  4. Preserve the local projection for investigation. Drop it only with the explicit additive rollback:

    Terminal window
    pnpm exec wrangler d1 execute publications-db-0 --remote \
    --file migrations/rollback/0049_workspace_support_tickets.sql --yes

Dropping the table removes only the local projection and does not close, edit, or delete GitHub issues. Reapply migration 0049 before restoring the capture/page paths.