Repository navigation
fix(snowflake): key the semantic catalog cache by database, not schema alone - #33926
Conversation
…a alone The schema-wide semantic catalog was cached under the schema name only, and the cache outlives a database: with threads=1 every database runs on the same thread. A multi-database run therefore served a later database's same-named schema the catalog of an earlier one, so its semantic views ingested with 0 metrics, dimensions and measures, and no warning.
✅ PR checks passedThe linked issue has a description and all required Shipping project fields set. Thanks! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
✅ Playwright Results — workflow succeededValidated commit ✅ 111 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky PerformanceBlocking targets: ✅ met · Optimization targets: 🟡 in progress Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting. 🕒 Full workflow signal wall (to summary) 48m 39s ⏱️ Max setup 4m 27s · max shard execution 12m 39s · max shard-job elapsed before upload 18m 22s · reporting 5s 🌐 102.07 requests/attempt · 1.79 app boots/UI scenario · 0.00% common-shard skew Optimization targets still in progress:
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Code Review ✅ Approved🟡 Medium risk · The cache key now scopes Snowflake semantic catalogs by database and schema. Keys the semantic catalog cache by OptionsDisplay: compact → Counting what did not apply, without listing it. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
|
🚦 Removed from the merge queue —
|
…a alone (#33926) (#34000) The schema-wide semantic catalog was cached under the schema name only, and the cache outlives a database: with threads=1 every database runs on the same thread. A multi-database run therefore served a later database's same-named schema the catalog of an earlier one, so its semantic views ingested with 0 metrics, dimensions and measures, and no warning. (cherry picked from commit 70d707b) Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>



Describe your changes:
Fixes #33927
When a Snowflake ingestion covers several databases that share a schema name (e.g.
DB_A.MARTSandDB_B.MARTS), the semantic views in the later database are ingested with 0 metrics, dimensions and measures, and nothing is logged. Ingesting that database on its own works._semantic_catalogcached the schema-wide catalog under the schema name only. The cache is never cleared between databases, and withthreads: 1every database runs on the same thread. SoDB_B.MARTSgotDB_A.MARTS's catalog, and every view lookup missed.(database, schema), using the topology context's database.Type of change:
Tests:
Use cases covered
Unit tests
yield_table_metricsone. Both fail without the fix.Ingestion integration tests
🤖 Generated with Claude Code
The PR appears safe to merge; no new findings were identified in its changes since the previous review.
Summary
The PR keys Snowflake’s semantic catalog cache by database and schema instead of schema alone, and adds regression tests for same-named schemas across databases.
Reviews (2) · Last reviewed commit: "Merge branch 'main' into fix/snowflake-s..."