Skip to content

Fixes #32377: cap pygtrie below 2.6 so import airflow works on Python 3.10 - #32378

Merged
ulixius9 merged 1 commit into
mainfrom
fix/pygtrie-2.6-py310-import
Sep 2, 2026
Merged

ulixius9 merged 1 commit into
mainfrom
fix/pygtrie-2.6-py310-import

Conversation

@Khairajani

Copy link
Copy Markdown
Contributor

Describe your changes:

Fixes #32377

py-tests has been red on the Python 3.10 lane since 2026-09-01T11:06Z, taking py-tests-status with it and evicting every PR that touches ingestion/ from the merge queue:

ImportError: cannot import name 'Connection' from 'airflow.models'

No code change caused it. pygtrie 2.6.0 was published at 11:06:08Z; it annotates with typing.Self (PEP 673, 3.11+) but declares no requires-python floor, so pip and uv install it on 3.10, where it is unimportable:

$ python3.10 -c "import pygtrie"
AttributeError: module 'typing' has no attribute 'Self'   # pygtrie.py:83, `def __copy__(self) -> _t.Self`
$ python3.13 -c "import pygtrie"   # fine

apache-airflow-core and apache-airflow-task-sdk both require pygtrie>=2.5.0 with no ceiling, and airflow._shared.logging.structlog imports it at module level — reached from airflow/__init__.py via configuration.initialize_secrets_backends(), which is why the AttributeError surfaces as a missing Connection. 3.11 and 3.12 are unaffected because they have typing.Self; 3.10 is MAIN_PYTHON_VERSION. Full chain and the package-diff evidence are in the issue.

This caps pygtrie<2.6 in VERSIONS and applies it in two places:

  • the airflow plugin extra, alongside the existing transitive pins — covers the Airflow image;
  • the test extra, which is where CI actually pulls Airflow in (uv pip install "ingestion[all]" then "ingestion[test]"; all-dev-env and all both exclude the airflow plugin).

ingestion/airflow-constraints-3.3.1.txt:583 already pins pygtrie==2.5.0, but the unit-test install path does not apply that constraints file — which is exactly why it did not protect CI.

The cap is temporary. It can be dropped once pygtrie declares requires-python>=3.11 (or switches to typing_extensions.Self), or Airflow caps the dependency, or 3.10 support is dropped. The comment in VERSIONS says so.

Type of change:

  • Bug fix

High-level design:

N/A — one-line dependency cap.

Tests:

Use cases covered

  • ingestion[test] resolves on Python 3.10 with a pygtrie that is actually importable
  • import airflow works again, so the Airflow connector unit tests and integration shard-3 collect

Unit tests

  • Not applicable — this is a dependency constraint; the verification is the resolver and CI itself (see below).

Backend integration tests

  • Not applicable (no backend API changes).

Ingestion integration tests

  • Not applicable (no ingestion code changes).

Playwright (UI) tests

  • Not applicable (no UI changes).

Manual testing performed

Full resolver dry-run of the affected extra on Python 3.10 — 421 packages, no conflict:

$ python3.10 -m pip install --dry-run --ignore-installed "./ingestion[test]"
  pygtrie: 2.5.0
  apache-airflow: 3.3.1
  apache-airflow-core: 3.3.1
  apache-airflow-task-sdk: 1.3.1

Before the cap the same resolution selects pygtrie 2.6.0, which is the failure. Also confirmed directly that 2.5.0 imports on 3.10 and 2.6.0 does not, and that 2.6.0 imports fine on 3.13.

This PR's own py-tests run on 3.10 is the real regression test.

UI screen recording / screenshots:

Not applicable.

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • My PR is linked to a GitHub issue via Fixes #<issue-number> above.
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: not applicable.
  • For UI changes: not applicable.
  • I have added tests (unit / integration / Playwright as applicable) and listed them above.

…thon 3.10

pygtrie 2.6.0 (released 2026-09-01T11:06Z) annotates with typing.Self, which
only exists on 3.11+, yet publishes no requires-python floor -- so pip and uv
happily install it on 3.10, where `import pygtrie` raises

    AttributeError: module 'typing' has no attribute 'Self'

apache-airflow-core and apache-airflow-task-sdk both require pygtrie>=2.5.0
with no ceiling, and airflow._shared.logging.structlog imports it at module
level. That module is reached from airflow/__init__.py through
configuration.initialize_secrets_backends(), so the AttributeError surfaces as

    ImportError: cannot import name 'Connection' from 'airflow.models'

which took out every py-tests run that resolved dependencies after 11:06Z on
the 3.10 lane -- unit tests and integration shard-3 alike. 3.11 and 3.12 were
unaffected because they have typing.Self.

Cap it in VERSIONS and apply it both to the airflow plugin extra and to the
test extra, which is where CI actually pulls Airflow in. Drop the cap once
pygtrie declares its floor or 3.10 support is dropped.
@Khairajani
Khairajani requested a review from a team as a code owner September 1, 2026 14:20
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot

gitar-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Caps pygtrie<2.6 in dependency pins to fix Python 3.10 import failures caused by pygtrie 2.6.0 using typing.Self without declaring a requires-python floor. The fix is applied to both the airflow plugin and test extras, restoring import airflow functionality and unblocking the Python 3.10 test lane. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@Khairajani Khairajani added safe to test Add this label to run secure Github workflows on PRs skip-pr-checks Bypass PR metadata validation check labels Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Playwright Results — workflow succeeded

Validated commit b99bb383a45fef34fc0ef7e23e336550f3af7bf9 in Playwright run 33519046276, attempt 1.

✅ 109 passed · ❌ 0 failed · 🟡 1 flaky · ⏭️ 0 skipped · 🧰 0 lifecycle flaky

Performance

Blocking targets: ✅ met · Optimization targets: 🟡 in progress

Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting.

🕒 Full workflow signal wall (to summary) 50m 21s

⏱️ Max setup 4m 22s · max shard execution 15m 18s · max shard-job elapsed before upload 21m 14s · reporting 4s

🌐 219.77 requests/attempt · 1.80 app boots/UI scenario · 0.00% common-shard skew

Optimization targets still in progress:

  • Browser traffic was 219.77 requests per attempt (convergence target: fewer than 200).
  • Application boot ratio was 1.8 per UI scenario (219 boots / 122 scenarios; convergence target: at most 1).
Shard Passed Failed Flaky Skipped Lifecycle failed Lifecycle flaky
✅ Shard chromium-01 46 0 0 0 0 0
🟡 Shard ingestion-01 28 0 1 0 0 0
✅ Shard ingestion-02 35 0 0 0 0 0
🟡 1 flaky test(s) (passed on retry)
  • Pages/TestSuiteDetailsPage.spec.ts › Add test case modal on Test Suite details page - filters and select (shard ingestion-01, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@sonarqubecloud

sonarqubecloud Bot commented Sep 1, 2026

Copy link
Copy Markdown

@ulixius9
ulixius9 added this pull request to the merge queue Sep 1, 2026
Merged via the queue into main with commit 0fab248 Sep 2, 2026
226 of 236 checks passed
@ulixius9
ulixius9 deleted the fix/pygtrie-2.6-py310-import branch September 2, 2026 01:59
Khairajani added a commit that referenced this pull request Sep 2, 2026
…ks on Python 3.10 (#32432)

* fix(ingestion): cap pygtrie below 2.6 so `import airflow` works on Python 3.10 (#32378)

pygtrie 2.6.0 (released 2026-09-01T11:06Z) annotates with typing.Self, which
only exists on 3.11+, yet publishes no requires-python floor -- so pip and uv
happily install it on 3.10, where `import pygtrie` raises

    AttributeError: module 'typing' has no attribute 'Self'

apache-airflow-core and apache-airflow-task-sdk both require pygtrie>=2.5.0
with no ceiling, and airflow._shared.logging.structlog imports it at module
level. That module is reached from airflow/__init__.py through
configuration.initialize_secrets_backends(), so the AttributeError surfaces as

    ImportError: cannot import name 'Connection' from 'airflow.models'

which took out every py-tests run that resolved dependencies after 11:06Z on
the 3.10 lane -- unit tests and integration shard-3 alike. 3.11 and 3.12 were
unaffected because they have typing.Self.

Cap it in VERSIONS and apply it both to the airflow plugin extra and to the
test extra, which is where CI actually pulls Airflow in. Drop the cap once
pygtrie declares its floor or 3.10 support is dropped.

(cherry picked from commit 0fab248)

* fix(lint): grandfather PLC0415 in test_get_dagbag so py-checkstyle passes on 2.0

The file was backported from main, where PLC0415 (import-outside-top-level) sits
in the global ruff ignore list, so its three function-local imports carry no
suppression. On this branch the rule is only waived through per-file-ignores that
do not cover openmetadata-airflow-apis/, so `make py_format_check` fails on every
PR against 2.0, unrelated to the change under review.

The imports are deliberate -- each test patches utils.settings before importing
the module under test -- so suppress rather than hoist, matching how the sibling
airflow-apis tests already do it (see test_deploy.py:60) and how pyproject.toml
documents grandfathering existing violations. Added with `ruff check --add-noqa`;
`make py_format` cannot fix PLC0415 on its own since the rule has no autofix.
Khairajani added a commit that referenced this pull request Sep 2, 2026
…thon 3.10 (#32378) (#32431)

pygtrie 2.6.0 (released 2026-09-01T11:06Z) annotates with typing.Self, which
only exists on 3.11+, yet publishes no requires-python floor -- so pip and uv
happily install it on 3.10, where `import pygtrie` raises

    AttributeError: module 'typing' has no attribute 'Self'

apache-airflow-core and apache-airflow-task-sdk both require pygtrie>=2.5.0
with no ceiling, and airflow._shared.logging.structlog imports it at module
level. That module is reached from airflow/__init__.py through
configuration.initialize_secrets_backends(), so the AttributeError surfaces as

    ImportError: cannot import name 'Connection' from 'airflow.models'

which took out every py-tests run that resolved dependencies after 11:06Z on
the 3.10 lane -- unit tests and integration shard-3 alike. 3.11 and 3.12 were
unaffected because they have typing.Self.

Cap it in VERSIONS and apply it both to the airflow plugin extra and to the
test extra, which is where CI actually pulls Airflow in. Drop the cap once
pygtrie declares its floor or 3.10 support is dropped.

(cherry picked from commit 0fab248)
Rohit0301 pushed a commit that referenced this pull request Sep 3, 2026
…thon 3.10 (#32378)

pygtrie 2.6.0 (released 2026-09-01T11:06Z) annotates with typing.Self, which
only exists on 3.11+, yet publishes no requires-python floor -- so pip and uv
happily install it on 3.10, where `import pygtrie` raises

    AttributeError: module 'typing' has no attribute 'Self'

apache-airflow-core and apache-airflow-task-sdk both require pygtrie>=2.5.0
with no ceiling, and airflow._shared.logging.structlog imports it at module
level. That module is reached from airflow/__init__.py through
configuration.initialize_secrets_backends(), so the AttributeError surfaces as

    ImportError: cannot import name 'Connection' from 'airflow.models'

which took out every py-tests run that resolved dependencies after 11:06Z on
the 3.10 lane -- unit tests and integration shard-3 alike. 3.11 and 3.12 were
unaffected because they have typing.Self.

Cap it in VERSIONS and apply it both to the airflow plugin extra and to the
test extra, which is where CI actually pulls Airflow in. Drop the cap once
pygtrie declares its floor or 3.10 support is dropped.
shrabantipaul-collate pushed a commit to PRADDZY/OpenMetadata that referenced this pull request Sep 3, 2026
…thon 3.10 (open-metadata#32378)

pygtrie 2.6.0 (released 2026-09-01T11:06Z) annotates with typing.Self, which
only exists on 3.11+, yet publishes no requires-python floor -- so pip and uv
happily install it on 3.10, where `import pygtrie` raises

    AttributeError: module 'typing' has no attribute 'Self'

apache-airflow-core and apache-airflow-task-sdk both require pygtrie>=2.5.0
with no ceiling, and airflow._shared.logging.structlog imports it at module
level. That module is reached from airflow/__init__.py through
configuration.initialize_secrets_backends(), so the AttributeError surfaces as

    ImportError: cannot import name 'Connection' from 'airflow.models'

which took out every py-tests run that resolved dependencies after 11:06Z on
the 3.10 lane -- unit tests and integration shard-3 alike. 3.11 and 3.12 were
unaffected because they have typing.Self.

Cap it in VERSIONS and apply it both to the airflow plugin extra and to the
test extra, which is where CI actually pulls Airflow in. Drop the cap once
pygtrie declares its floor or 3.10 support is dropped.

This branch was previously deployed

1 inactive deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs skip-pr-checks Bypass PR metadata validation check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

py-tests red on Python 3.10: pygtrie 2.6.0 breaks import airflow (cannot import name 'Connection' from 'airflow.models')

3 participants