feat/jira-rich-migration #177
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/jira-rich-migration"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Add migration_source_resources — the resource-level sibling of migration_source_issues — giving every imported resource (comment, attachment, worklog, …) a durable, atomically-claimed binding per destination project. SourceResourceReservations mirrors the issue claim lifecycle (reserved → creating_remote → bound, stale reclaim, ambiguous probing, deterministic-failure classification) and adds a metadata_only terminal state plus source_checksum drift detection: a resource the source edited after import is recorded as a conflict, never silently overwritten. import_mode gains 'enrich' alongside missing|reconcile: the item resolves the work packages a previous import created (durable binding → wp_ids → unambiguous [KEY] marker — never creates tickets), then applies the resource importers. Source issues with no destination ticket are reported in enrich_new. Per-issue enrich_* counters plus per-resource-kind {kind}_* subcounts record the outcome; unresolved ambiguous resolutions and unsettled resource claims land the item on deferred — never a silent completion. Comments are the first category: JiraMigrationSource::fetchComments paginates the declared total to its end (truncated listings throw, retryable), ADF bodies reuse the description flattener, and JiraCommentImporter writes each comment as an OpenProject journal with an explicit attribution preamble (the 'Jira comment {id}' marker doubles as the ambiguous-outcome probe — adopt-or-recreate on the next pass). OpenProject authorship is never falsified. The wizard offers 'Enrich existing tickets' when a destination has prior imports; the run page reports the enrich and comments counters; the capability matrix marks comments partial and the remaining resource kinds as planned.Extend the enrich/missing Jira migration with user-identity resolution and three more resource categories behind the migration_source_resources claim protocol: - migration_user_mappings records every seen source identity per org + source instance (stable source_user_ref, sha256 source_email_hash — raw emails never stored). MigrationUserResolver resolves in a strict order: verified mapping row, then an unambiguous verified-email match scoped to the migration's own organization, then unresolved. No name-only matching, no cross-org matches, and unresolved identities are never attributed to the operator — they persist for admin review via POST /migrations/{id}/user-mappings/{mapping} and defer the item. - JiraWorklogImporter books each worklog as an OpenProject time entry (seconds to ISO-8601 duration, started to spent_on, flattened comment with the Jira worklog marker for ambiguous-outcome probing). The activity resolves from devworkbase.migrations.default_time_activity against the live catalog — an unresolvable name reports worklogs_unresolved:activity instead of guessing. Unresolved authors skip with user_unresolved; zero-second or undated worklogs record metadata_only. - JiraWatcherImporter adds resolved watchers through the ticket's watcher list (the adopt/probe oracle), claiming each {issue}:{user} link durably; unresolved watchers are reported, never guessed. - Assignees apply in missing-mode creation and enrich-mode updates only through a resolved mapping; the current assignee_id is compared first so unchanged tickets are never re-saved, and unresolvable assignees are reported with the destination assignee untouched. - OpenProjectClient gains listWorkPackageWatchers, addWorkPackageWatcher, removeWorkPackageWatcher and assignee_id in the normalized work-package shape. - Item/run counters extend per the {prefix}_* contract: worklogs_*, watchers_* plus the new unresolved map suffix on every prefix, assignee_applied/assignee_unresolved and users_unresolved; totals surface in the controller serialization and the run page. Also merges the shadowed duplicate devworkbase.migrations config key (stale_item_minutes was unreachable) and adds default_time_activity. New suites: MigrationUserMappingTest, MigrationWorklogTest.Enrich mode now carries three more Jira resource categories through the shared migration_source_resources claim contract: - Issue history: GET /issue/{id}/changelog (offset-paginated against the declared total; Data Center falls back to GET /issue/{id}?expand=changelog). Every change-group lands as a metadata_only history_event row carrying the structured event verbatim, and one labeled "Imported Jira history" journal renders the full chronological record (author, timestamp, field, from->to) — OpenProject cannot create backdated or author-attributed journals, so real actors are declared in text, never impersonated. A dedicated {issue}:journal row binds the journal id; the marker probe adopts crash-committed journals instead of duplicating them. - Custom fields: the issue search requests *navigable fields and GET /field supplies labels and schema types. Values matching an enabled, writable, type-compatible destination field (validated through the ticket's own schema before any write) are applied via updateWorkPackage — scalar formats as attributes, option types via the resolved custom_option link. Incompatible types, unmapped fields and user references are preserved verbatim on metadata_only rows and reported in custom_fields_unsupported; global destination fields are never created. - Boards: the board listing shared with sprint discovery records each board verbatim ({id, name, type, location}) as a metadata_only board row at import time and on enrich passes; native destination boards are not created and the capability matrix reports partial. MigrationSourceResource gains a JSON metadata column for verbatim preservation; the runner folds the new history_/custom_fields_/ boards_ counter prefixes into payloads, audit meta and the status endpoint, and Show.tsx renders them.Fresh Jira imports now run the full per-ticket resource pipeline through a shared importJiraResources() used by both missing-mode import and enrich: comments, attachments, worklogs, watchers, issue history and custom fields land behind the resource-claim contract after every ticket is bound — never before its durable migration_source_issues row, so a child-resource failure can never re-create the work package (wp_done + reservations own retry). Resource report maps are rebuilt per attempt so a resolved ref can no longer pin an item deferred/failed forever. Attachment identity is stronger than filename+size: every upload stamps 'Jira attachment {id}' plus the source instance into the attachment description — OpenProject persists the upload metadata.description and echoes it back on list — so the ambiguous probe adopts on immutable source identity. Name+size survives only as a last-resort single-candidate fallback, recorded as adopted_via=name+size on the row; a >1 collision never auto-adopts. The listing pages to the collection's declared total and throws on truncation — an unverifiable listing stays ambiguous rather than blind-recreating — and a zero-candidate probe re-lists once after a delayed-visibility grace before 'not uploaded' is concluded. The assignee field applies under a strict provenance policy: the source assignee may write onto an empty destination value or over a value the importer itself recorded via the item payload's importer_owned marker; a destination value differing from both is a user change — recorded under assignee_conflicts, never overwritten. Unresolvable identities are reported, never defaulted to the operator. The migration page gains a Source user mappings section: unresolved Jira identities list with suggested matches (verified email-hash or unique display-name — never auto-resolved), a member select, and resolution through the existing admin-gated endpoint; resolved mappings take effect on the next pass.Two correctness fixes in the Jira attachment source-identity contract: 1. probeHits no longer auto-adopts on name+size alone. Adoption now requires the verified marker — `Jira attachment {id} · migration source {baseUrl}` — matching BOTH the attachment id and the normalized source instance. A marker stamped by a different Jira instance is a colliding identity claim (conflict, never adopted); a marker-less name+size or md5-digest twin is an unverifiable_candidate — surfaced ambiguous for reconciliation, adopted_via never stamped on resemblance. The upload's content md5 is recorded on the reservation row and corroborates marker hits (adopted_via=marker+digest) — a secondary signal, never a replacement for identity. 2. recordCreateFailure now classifies by transmission class: pre-transmission failures (4xx, validation, connect/DNS refusals) are failed and retryable; post-transmission-uncertain (5xx, timeout mid/after send, connection drop) is ambiguous and must NOT auto-retry-create. Two empty listings cannot prove a timed-out create never committed, so probeResolve no longer re-creates for creating_remote/ambiguous rows on zero candidates — the row is held for a verified-marker adoption or an explicit reconcileBind. Stale creating_remote reclaims stamp ambiguous, never the provably-pre-write reserved state. New probe verdict channel: a claim's candidate callable may throw ResourceProbeVerdict (conflict/unverifiable) for verdicts a candidate list cannot express, and receives the reservation row so stamped provenance can corroborate candidates. reconcileBind is the service-level primitive for operator-authorized adoption — no HTTP repair action exists for resource rows yet, so ambiguity persists until an authorized path confirms it. listWorkPackageAttachments exposes the OP digest; the fake gains connection-refused (pre-write) and vanish-timeout (uncertain) upload failure shapes. Tests cover marker-verified adopt, wrong-instance conflict, unverifiable twins, digest-as-secondary, uncertain no-recreate, retry classes, explicit reconciliation, and a real two-worker attachment upload convergence.