Repository navigation
Conversation
When a Grafana dashboard row is collapsed the API nests child panels inside the row panel's `panels` field instead of the top-level `dashboard.panels` list. The ingestion connector iterated only the top-level list, so all panels under a collapsed row were silently skipped — they never appeared as charts and were excluded from lineage. Changes: - `GrafanaPanel`: add optional `collapsed` and `panels` fields to represent the nested structure returned by the Grafana API; call `model_rebuild()` to resolve the self-referential forward reference. - `GrafanaSource._flatten_panels()`: new static helper that walks the panel list and replaces collapsed row panels with their children. Expanded rows (where child panels are already at the top level) are left unchanged. - `yield_dashboard_chart` / `yield_dashboard_lineage_details`: call `_flatten_panels` before iterating so both code paths see every panel. - Unit tests: added `DASHBOARD_WITH_COLLAPSED_ROW` fixture and three test methods covering collapsed rows, expanded rows, and empty input. Prepared with AI assistance (Claude Code, Anthropic), reviewed for correctness before submission.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hey @zerafachris, thanks for the contrib! The production change looks good, but the regression tests currently exercise This leaves an important gap: if the Could you add behavioral coverage for both consumers?
Also, since you are touching those tests, can you please migrate them from unittest to pytest? Thanks |
…ineage; migrate to pytest
|
Thanks @IceS2! I've pushed a follow-up commit (0c7db36) that addresses all three points:
|
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
4 similar comments
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Applied |
Builds on PR open-metadata#31611 (open-metadata#31474): - reformat with ruff (supersedes the black commits) so py-checkstyle passes; fix F401/B905 - models.py: collapsed/panels as `| None` (drop Optional); keep GrafanaPanel.model_rebuild - metadata.py: parameterized (%) logging — no f-strings in logger calls - test_grafana.py: restore test_complete_json_parsing targeted assertions (nested targets, format 0-vs-None, per-target datasource, meta fields) - test_grafana_simple.py: migrate unittest -> pytest
…ion) The 'from tests.unit.topology.dashboard.fixtures...' import failed under the repo's pytest invocation (ModuleNotFoundError: No module named 'tests.unit') — it was the only 'from tests.*' import in the suite. Inline the collapsed-row panels in the flatten test (matching test_grafana.py) and drop the now-unused DASHBOARD_WITH_COLLAPSED_ROW fixture.
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
The Python checkstyle failed. Please run You can install the pre-commit hooks with |
Code Review ✅ Approved 1 resolved / 1 findingsGrafana dashboard ingestion now surfaces panels nested inside collapsed rows by adding recursive panel fields to the model and flattening collapsed-row children before chart and lineage extraction. Comprehensive fixtures and unit tests cover collapsed rows, expanded rows, and edge cases. No issues found. ✅ 1 resolved✅ Quality: test_complete_json_parsing weakened from full-object equality
OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
a18e124



Why
Fixes #31474
When a Grafana dashboard row is collapsed, the Grafana REST API nests its child panels inside the row panel's own
panelsfield instead of the top-leveldashboard.panelslist. The connector iterated only the top-level list, so every panel under a collapsed row was silently skipped — those panels never appeared as charts and were excluded from lineage.Root cause
GrafanaPanelhad nocollapsedorpanelsfields, so the nested panels were discarded during JSON deserialisation. Even if they had been preserved,yield_dashboard_chartandyield_dashboard_lineage_detailsiterateddashboard.panelsdirectly without unwrapping row children.What changed
grafana/models.pycollapsed: Optional[bool]andpanels: Optional[List["GrafanaPanel"]]toGrafanaPanel; addedGrafanaPanel.model_rebuild()to resolve the self-referential forward referencegrafana/metadata.pyGrafanaSource._flatten_panels()— a static helper that replaces collapsed row panels with their children, leaving expanded rows (children already at top level) unchanged; updatedyield_dashboard_chartandyield_dashboard_lineage_detailsto pass the panel list through this helper before iteratingfixtures/grafana_fixtures.pyDASHBOARD_WITH_COLLAPSED_ROWfixture covering: a top-level panel, a collapsed row with two nested panels, an expanded row with no nested panels, and a top-level panel that follows the expanded rowtest_grafana_simple.pyBehaviour
Before: panels inside a collapsed Grafana row → silently missing from OpenMetadata charts and lineage.
After:
_flatten_panelssurfaces those panels; the rest of the pipeline handles them identically to top-level panels.Expanded rows are unaffected — when a row is expanded, Grafana already places child panels at the top level and the row's own
panelslist is empty, so_flatten_panelsis a no-op for that case.Testing
Unit tests added in
test_grafana_simple.pycover all three scenarios. Full integration test suite requires the OpenMetadata build environment; CI will validate.Prepared with AI assistance (Claude Code, Anthropic), reviewed for correctness before submission.
Greptile Summary
The PR updates Grafana ingestion to deserialize panels nested inside collapsed rows and process them as ordinary dashboard charts and lineage inputs.
Confidence Score: 5/5
The PR appears safe to merge.
No blocking failure remains.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart LR API[Grafana dashboard response] --> Parse[Deserialize dashboard panels] Parse --> Flatten[Replace collapsed rows with child panels] Flatten --> Charts[Create chart requests] Flatten --> Lineage[Extract dashboard lineage] Charts --> Catalog[OpenMetadata catalog] Lineage --> CatalogReviews (5): Last reviewed commit: "fix(grafana): make test_grafana_simple s..." | Re-trigger Greptile
Context used: