Duplicate Merge Workflow

Duplicate merge workflow

The duplicate detection and merge workflow (DASH-DATA-002) lets an authorized curator compare a candidate pair of publications and execute a human-confirmed, provenance-preserving merge. First-release policy — tombstone-preserving merges, alias lookup instead of HTTP redirects, forward-only repair — is recorded in ADR-0022 (docs/adr/0022-duplicate-merge-tombstone-alias-forward-only-repair.md), which answers the source packet’s open questions before implementation.

Candidate model

• Heuristic signal: the ingestion pipeline already flags duplicate_candidate review items (metadata review queue) from provider metadata, with compact evidence (candidate_count, candidates: [{id, similarity}]).
• Deterministic rules: assessDuplicateCandidatePair (@platform/utils, vocabulary in @platform/schemas) evaluates the first-release cases over a compact record summary: same_doi, same_normalized_title, title_year_match, author_overlap. Every rule reports an explicit match state with plain-language detail — candidate generation is reproducible and idempotent by construction, and the assessment is explainable evidence, never a quality score (ADR-0020).
• Tiering: likely (same DOI, or title+year with author overlap), possible (title match, or author overlap), unlikely (no rule matched). Tiers inform review priority; they never trigger automatic merges — automatic destructive merging is out of scope for v1.

Merge contract

• Preview (non-mutating): dashboard server function getPublicationMergePreview (apps/dashboard/src/server/functions/publications/merge.ts). It composes the existing /v1/detail and /v1/provenance reads for both records and returns: record summaries with provenance states, the deterministic assessment, an only-fill-empty plan over the split-view editable set (title, abstract, authors, publication_year — provider-owned catalogue fields stay compare-only per the split-view ADR), warnings (both_edited, provider_conflict, retraction_mismatch, visibility_mismatch, cross_shard), and a digest bound to the exact pair and workspace plus an issue timestamp.
• Execution: executePublicationMerge. Authorization follows the destructive-operations gate (workspace admin/owner, superadmin). Confirmation reuses the bulk-curation guardrails: the literal CONFIRM, the digest, and the preview TTL are validated server-side before any dispatch. Execution then re-reads both records and recomputes the fill plan — it never trusts preview-era values or client counts.
• Field fills flow through the existing /v1/patch edit model, so edited-data representations, edit counting, and provenance attribution are unchanged; the merge activity event and the lineage row record which values came from the merged-away record. No new provenance state and no second trust vocabulary.

What the merge does mechanically

worker-publication-write gains POST /v1/merge ({entity_id, canonical_doi, duplicate_doi}):

1. Verify the canonical record exists, live, in the workspace (404 otherwise, no writes).
2. Verify the duplicate exists; a tombstoned duplicate means the merge already ran — the response reports already_tombstoned and the retry converges.
3. Re-point publication_people links: insert only the links the canonical record lacks (INSERT OR IGNORE, union semantics) on the canonical shard.
4. Tombstone the duplicate on its own shard: delete its person links and set deleted_at (guarded by deleted_at IS NULL). The row and its original_data/edited_data are retained — merges never hard-delete.

deleted_at has no other writer on publications; list, stats, and timeseries reads already filter deleted_at IS NULL, so merged-away records leave those surfaces with no read-path rewrite.

Alias behavior (redirect policy)

• Dashboard reads: getPublication consults the lineage (best-effort, fail-open) and answers a merged-away id with an explicit {error: "merged", canonical_id} (HTTP 409) instead of stale duplicate content.
• Consumer API: unchanged in the first release — the merged-away id disappears from list surfaces; direct-lookup aliasing and consumer communication belong to the DASH-DATA-002 TKT-004 rollout.

Lineage ledger (AOA)

publication_merge_lineage (migration 0059, DB_0 compat → DB_6 target, topology-registered) holds one row per merged-away record with a pending → completed lifecycle, the actor, the curator note, resolved review items, link counts, and the pre-merge snapshot (canonical prior fields for filled fields, compact duplicate snapshot). The unique (entity_id, merged_away_id) identity plus per-step idempotent guards make execution convergent under retry: a crash in any window (before/after tombstone, before completion) is repaired by re-running the same command, and a repeat of a completed merge answers the explicit already_merged outcome.

Review items for the pair are resolved (fixed) with a merge reference, and the decision is written as an attributable publication.merged activity event naming the actor, pair, fills, and link counts.

Recovery and rollback

• Forward-only repair: there is no automatic un-merge in v1 — post-merge edits would make naive reversal ambiguous. The tombstoned row plus the lineage snapshot keep both pre-merge states reconstructable for deliberate, human-driven repair.
• Execution kill switch: set PUBLICATION_MERGE_EXECUTION_ENABLED=false on the dashboard worker to disable merge execution while candidate review and merge preview remain available.
• Full rollback: the alias check, preview, and execution are additive contracts; removing the merge UI route/contract restores the pre-merge surface, and lineage rows are retained for investigation.
• Search-projection cleanup for merged-away records is a known TKT-004 cleanup item (the existing hard-delete path has the same gap).