Repository navigation
fix(lineage): keep an edge writable after its pipeline is deleted [2.0] - #35315
Conversation
…4816) Ingestion copies an edge's stored pipeline onto every lineage write that carries none (SQL-parsed and query-log lineage), so those writes do not strip the pipeline a pipeline connector attached. addLineage and patchLineageEdge re-validated that pipeline with Include.NON_DELETED on every write. Once the pipeline was soft-deleted, for example because its dbt Cloud job was removed, every later write to the edge failed with "pipeline instance for <id> not found", nothing was saved, and the agent logged the same 404 on every run. Only a pipeline newly linked to an edge now has to exist and not be deleted. A pipeline the edge already references is kept when it was soft-deleted, and dropped when it was hard-deleted, since hard delete removes only PipelineLineage edges and other edges can still reference it.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
Code Review ✅ Approved🟡 Medium risk · Changing lineage writes to retain soft-deleted pipeline references and drop hard-deleted ones could leave stale or missing pipeline attribution on edges. Keeps lineage edges writable when their referenced pipeline has been soft-deleted, so unchanged PUT and PATCH writes succeed, and drops pipelines that were hard-deleted from an edge instead of failing the write. Adds VerificationRules No rules evaluated Functional validation Not enabledAuto-approval Not enabledOptionsDisplay: compact → Counting what did not apply, without listing it. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
backport of 5620522 on 2.0
Summary by Gitar
LineageDeletedPipelineITcovering PUT and PATCH on edges with deleted pipelinesThis will update automatically on new commits.