Skip to content

fix(snowflake): key the semantic catalog cache by database, not schema alone - #33926

Merged
ShaileshParmar11 merged 2 commits into
mainfrom
fix/snowflake-semantic-catalog-cache-per-database
Sep 24, 2026
Merged

ShaileshParmar11 merged 2 commits into
mainfrom
fix/snowflake-semantic-catalog-cache-per-database

Conversation

@Khairajani

@Khairajani Khairajani commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Describe your changes:

Fixes #33927

When a Snowflake ingestion covers several databases that share a schema name (e.g. DB_A.MARTS and DB_B.MARTS), the semantic views in the later database are ingested with 0 metrics, dimensions and measures, and nothing is logged. Ingesting that database on its own works.

_semantic_catalog cached the schema-wide catalog under the schema name only. The cache is never cleared between databases, and with threads: 1 every database runs on the same thread. So DB_B.MARTS got DB_A.MARTS's catalog, and every view lookup missed.

  • Key the cache by (database, schema), using the topology context's database.

Type of change:

  • Bug fix

Tests:

Use cases covered

  • Multi-database Snowflake run where two databases share a schema name: the later database's semantic views now get their metrics.

Unit tests

  • Added a catalog-level regression test and a yield_table_metrics one. Both fail without the fix.

Ingestion integration tests

  • Not applicable (covered by unit tests).

🤖 Generated with Claude Code

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no new findings were identified in its changes since the previous review.

Summary

The PR keys Snowflake’s semantic catalog cache by database and schema instead of schema alone, and adds regression tests for same-named schemas across databases.

  • The cache remains bounded and per-thread.
  • The new tests cover catalog lookup and metric emission in a multi-database run.

Reviews (2) · Last reviewed commit: "Merge branch 'main' into fix/snowflake-s..."

…a alone

The schema-wide semantic catalog was cached under the schema name only, and
the cache outlives a database: with threads=1 every database runs on the same
thread. A multi-database run therefore served a later database's same-named
schema the catalog of an earlier one, so its semantic views ingested with 0
metrics, dimensions and measures, and no warning.
@Khairajani
Khairajani requested a review from a team as a code owner September 24, 2026 06:11
@Khairajani
Khairajani requested a review from TeddyCr September 24, 2026 06:11
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR checks passed

The linked issue has a description and all required Shipping project fields set. Thanks!

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit f4c279f3016a1e32e4af2fcb44077db205dd627b in Playwright run 35965992210, attempt 1.

✅ 111 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 48m 39s

⏱️ Max setup 4m 27s · max shard execution 12m 39s · max shard-job elapsed before upload 18m 22s · reporting 5s

🌐 102.07 requests/attempt · 1.79 app boots/UI scenario · 0.00% common-shard skew

Optimization targets still in progress:

  • Application boot ratio was 1.79 per UI scenario (218 boots / 122 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
✅ Shard chromium-01 46 0 0 0 0 0
✅ Shard ingestion-01 32 0 0 0 0 0
✅ Shard ingestion-02 33 0 0 0 0 0

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot

gitar-bot Bot commented Sep 24, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

🟡 Medium risk · The cache key now scopes Snowflake semantic catalogs by database and schema.

Keys the semantic catalog cache by (database, schema) instead of schema alone, fixing multi-database Snowflake ingestions where shared schema names caused metric/dimension/measure data loss in later databases. Regression and unit tests added; no issues found.

Review coverage

📋 Rules No rules evaluated

🧪 Functional validation Not enabled · Set up

🤖 Auto-approval Not enabled · Set up

Options

Display: compact → Counting what did not apply, without listing it.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@Khairajani Khairajani added the safe to test Add this label to run secure Github workflows on PRs label Sep 24, 2026
@Khairajani
Khairajani added this pull request to the merge queue Sep 24, 2026
@sonarqubecloud

Copy link
Copy Markdown

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Sep 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🚦 Removed from the merge queue — failed_checks (2026-09-24T12:48:07Z)

Blocked the queue: playwright-summary

@ShaileshParmar11
ShaileshParmar11 added this pull request to the merge queue Sep 24, 2026
Merged via the queue into main with commit 70d707b Sep 24, 2026
188 of 199 checks passed
@ShaileshParmar11
ShaileshParmar11 deleted the fix/snowflake-semantic-catalog-cache-per-database branch September 24, 2026 19:47
Khairajani added a commit that referenced this pull request Sep 25, 2026
…a alone (#33926) (#34000)

The schema-wide semantic catalog was cached under the schema name only, and
the cache outlives a database: with threads=1 every database runs on the same
thread. A multi-database run therefore served a later database's same-named
schema the catalog of an earlier one, so its semantic views ingested with 0
metrics, dimensions and measures, and no warning.

(cherry picked from commit 70d707b)

Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Snowflake: semantic views in a same-named schema of a later database ingest with 0 metrics

3 participants