Skip to content

Drop the kms hook when the proprietary NVIDIA driver handles early KMS - #6791

Merged
dhh merged 5 commits into
omacom:quattrofrom
matjam:nvidia-drop-kms-hook
Aug 13, 2026
Merged

dhh merged 5 commits into
omacom:quattrofrom
matjam:nvidia-drop-kms-hook

Conversation

@matjam

@matjam matjam commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Fixes #6790.

install/hardware/nvidia.sh early-loads nvidia_drm (with modeset=1) for early KMS, but HOOKS still carries the kms hook, so autodetect pulls nouveau — and ~100 MB of its GSP firmware — into every initramfs for a driver that never runs. On a Limine UKI setup that means a 256 MB image where ~144 MB is normal, doubled again by the fallback history on /boot.

Change

omarchy_hooks.conf now filters kms out of HOOKS when both hold:

  • nvidia_drm is in MODULES — i.e. nvidia.conf, which sorts before this drop-in, configured proprietary early KMS;
  • every PCI display-class device (class 0x03xx in sysfs) has NVIDIA's vendor ID.

Hybrid iGPU+dGPU systems keep kms so the iGPU retains early KMS at the LUKS prompt. Non-NVIDIA systems are untouched. Because the conditional lives in the settings-package config, existing installs pick it up on their next settings update + initramfs rebuild — no migration needed.

Testing

  • Sourced with nvidia_drm in MODULES against real sysfs on an RTX 4090 system: kms dropped.
  • Sourced without nvidia_drm, and with MODULES unset under set -u: HOOKS unchanged, no errors.
  • Faked-sysfs hybrid tree (AMD 0x1002 iGPU + NVIDIA dGPU): kms kept.
  • Real-world result of the equivalent filter on the same machine (nvidia-open-dkms 610.57.04, Limine + UKI + LUKS): UKI 256,183,296 → 144,066,048 bytes; nouveau.ko and its firmware gone; the two nvidia-utils GSP blobs and all four nvidia modules retained; Plymouth still owns the LUKS prompt via nvidia_drm.

Filed by Claude Fable 5 via Claude Code.

🤖 Generated with Claude Code

install/hardware/nvidia.sh early-loads nvidia_drm (modeset=1) for early KMS,
but HOOKS still carried the kms hook, so autodetect pulled nouveau and
~100 MB of its GSP firmware into every initramfs for a driver that never
runs. On a Limine UKI setup that meant a 256 MB image where ~144 MB is
normal, doubled again by the fallback history on /boot.

Filter kms out of HOOKS when nvidia_drm is in MODULES (nvidia.conf sorts
before this drop-in) and every PCI display controller is NVIDIA. Hybrid
systems keep kms so the iGPU retains early KMS at the LUKS prompt.

Verified on an RTX 4090 (nvidia-open-dkms 610.57.04): UKI shrinks
256,183,296 -> 144,066,048 bytes, nouveau and its firmware gone, the
nvidia-utils GSP blobs and all four nvidia modules retained, Plymouth
still owns the LUKS prompt via nvidia_drm.

Fixes omacom#6790

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings August 13, 2026 07:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Conditionally removes the kms initramfs hook on NVIDIA-only systems while preserving it for hybrid graphics.

Changes:

  • Detects proprietary NVIDIA early KMS configuration.
  • Retains kms when a non-NVIDIA display controller exists.

Tip

If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.

Suppressed comments (1)

etc/mkinitcpio.conf.d/omarchy_hooks.conf:19

  • The kms string literal should be quoted to follow the Bash comparison convention documented in AGENTS.md:17.
      [[ $_omarchy_hook == kms ]] || _omarchy_hooks+=("$_omarchy_hook")

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Quote fixed string literals in the [[ ]] comparisons per AGENTS.md, and
read the PCI tree through OMARCHY_PCI_DEVICES_PATH, the same seam
bin/omarchy-hw-nvidia already uses.

Require a positively identified NVIDIA display controller before dropping
kms: an empty or unreadable PCI tree previously counted as "no non-NVIDIA
GPU" and would have dropped the hook. Unexpected trees now keep kms.

Cover the conditional in test/shell.d/nvidia-kms-hook-test.sh: nvidia-only,
hybrid, no nvidia_drm, MODULES unset under set -u, audio-function-only,
empty tree, and a device directory missing its sysfs attributes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 13, 2026 07:10
@matjam

matjam commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the review in b1058de:

  • Fixed string literals in the [[ ]] comparisons are now quoted per AGENTS.md, with the class wildcard left active ("0x03"*).
  • Added test/shell.d/nvidia-kms-hook-test.sh covering nvidia-only, hybrid, no nvidia_drm, MODULES unset under set -u, an NVIDIA audio function with no display controller, an empty PCI tree, and a device directory missing its sysfs attributes.
  • Writing the empty-tree test exposed a real gap: an empty or unreadable PCI tree counted as "no non-NVIDIA GPU" and would have dropped kms. The conditional now requires a positively identified NVIDIA display controller, and reads the tree through OMARCHY_PCI_DEVICES_PATH — the same seam bin/omarchy-hw-nvidia uses.

Filed by Claude Fable 5 via Claude Code.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

etc/mkinitcpio.conf.d/omarchy_hooks.conf:15

  • This skips an unreadable PCI function without marking detection as inconclusive. If a readable NVIDIA GPU exists beside that function, _omarchy_nvidia_gpu becomes true while _omarchy_other_gpu stays false, so kms is removed even though the code could not verify that every display controller is NVIDIA. Treat an unreadable entry conservatively so the documented fallback actually applies.
    [[ -r $_omarchy_pci/class && -r $_omarchy_pci/vendor ]] || continue

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (2)

etc/mkinitcpio.conf.d/omarchy_hooks.conf:15

  • Unreadable entries are skipped, so a tree with one readable NVIDIA GPU plus an unreadable second device still removes kms. That does not establish that every display controller is NVIDIA and defeats the documented fail-safe if the unreadable device is another GPU. Treat an unreadable entry as unknown so the filter keeps kms; the test should also cover this mixed case.
    [[ -r $_omarchy_pci/class && -r $_omarchy_pci/vendor ]] || continue

test/shell.d/nvidia-kms-hook-test.sh:38

  • This sources the host's /etc/vconsole.conf under set -u, making the new test machine-dependent. The repository itself notes that this file may contain only KEYMAP (migrations/1784476564.sh:11); in that valid case the later ${XKBLAYOUT%%,*} expansion aborts before any NVIDIA assertion runs. Initialize the unrelated mkinitcpio inputs before sourcing the config.
  OMARCHY_PCI_DEVICES_PATH="$tmp_dir/devices" bash -uc "
    $modules_decl

A device whose class/vendor attributes cannot be read could be another
GPU, so skipping it let a readable NVIDIA GPU beside it drop kms without
having verified the whole tree. Count it as a non-NVIDIA sighting so kms
stays, and cover the mixed case in the test.

The test also sourced the host's /etc/vconsole.conf under set -u, where a
valid KEYMAP-only file makes the XKBLAYOUT expansion fail in the subshell
and ties the result to the machine running it. Predefine XKBLAYOUT and
FILES before sourcing the config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 13, 2026 07:22
@matjam

matjam commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

The suppressed review comments were both right, addressed in 94ed33e:

  • An unreadable PCI device next to a readable NVIDIA GPU previously still dropped kms — the skip treated "can't read it" as "not a GPU". Unreadable entries now count as inconclusive, so kms stays, and the test covers the mixed case.
  • The test sourced the host's real /etc/vconsole.conf under set -u, where a valid KEYMAP-only file breaks the XKBLAYOUT expansion and makes the result machine-dependent. XKBLAYOUT and FILES are now predefined before sourcing, so the test is hermetic.

Filed by Claude Fable 5 via Claude Code.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

dhh and others added 2 commits August 13, 2026 00:33
The settings package deploys the new omarchy_hooks.conf conditional, but
nothing rebuilds the initramfs when only a mkinitcpio drop-in changes, so
existing NVIDIA-only installs would carry nouveau's ~100 MB of GSP
firmware until their next kernel update. Following the precedent of
1784476564, add a migration that rebuilds via limine-mkinitcpio — once
per machine, and only where evaluating the installed drop-ins shows the
conditional actually dropped kms, so hybrid machines, non-NVIDIA
machines, and user-edited configs are left alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A later-sorting drop-in that resets MODULES outright (as
surface_device_modules.conf does) would strip nvidia_drm after kms was
already dropped. Every machine Omarchy writes such a file for is hybrid
Intel and keeps kms through the PCI scan, but that is worth stating so
the invariant is not broken by accident.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings August 13, 2026 07:34
@dhh

dhh commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Reviewed this against the codebase, pushed two fixup commits (e66ccc7, 5d5cc06), and ran an adversarial codex review over the full diff.

What I checked and found sound

  • Driver detection. Gating on nvidia_drm in MODULES is the right signal: install/hardware/nvidia.sh writes /etc/mkinitcpio.conf.d/nvidia.conf for both nvidia-open-dkms and nvidia-580xx-dkms, and never for nouveau. nvidia.conf sorts before omarchy_hooks.conf, so the modules are visible when the conditional runs.
  • Hybrid graphics. The PCI scan (etc/mkinitcpio.conf.d/omarchy_hooks.conf:20) requires a positively identified NVIDIA display controller and no non-NVIDIA one, using class 0x03* — which covers both VGA (0x0300) iGPUs and 3D-controller (0x0302) dGPUs — so an Intel/AMD iGPU keeps kms and its early KMS at the LUKS prompt. Empty/unreadable trees fall back to keeping kms. Re-evaluated on every rebuild, so a GPU swap self-heals.
  • HOOKS manipulation. Array filtering with exact element comparison, no sed on the HOOKS line — no word-boundary or substring hazards, and a duplicate kms would be removed cleanly.
  • Fresh installs. nvidia.conf lands during omarchy-apply-hardware, before any subsequent initramfs build reads the drop-ins.
  • No conflicts with omarchy_resume.conf (HOOKS+=(resume), sorts later, appends) or thunderbolt_module.conf.

What was missing: existing installs never rebuild

omarchy_hooks.conf ships as a backup=() file in omarchy-settings, and nothing regenerates the initramfs when only a mkinitcpio drop-in changes — so every existing NVIDIA install would keep the ~100 MB image (doubled in the Limine fallback history) until its next kernel update. The repo's precedent for exactly this situation is migrations/1784476564.sh, which rebuilt after changing this same file. Pushed:

  • migrations/1786605598.sh — rebuilds via limine-mkinitcpio once per machine (marker in /var/lib/omarchy/migrations/, same pattern as migrations/1786482992.sh), and only where sourcing the installed drop-ins in mkinitcpio's order shows kms actually dropped out — so hybrid machines, non-NVIDIA machines, and user-edited configs (packaged update parked as .pacnew) skip it.
  • test/shell.d/nvidia-kms-migration-test.sh — covers rebuild + marker, cross-user idempotency, interrupted-rebuild retry, hybrid skip, pre-conditional config skip, and missing limine-mkinitcpio/nvidia.conf skips.

Codex review (xhigh) findings and disposition

  1. Later drop-in resets MODULES: install/hardware/fix-surface-keyboard.sh:20 writes surface_device_modules.conf with MODULES=(...) (assignment, not +=), which sorts after this file and would strip nvidia_drm after kms was already dropped. Unreachable today — every machine Omarchy writes that file for is hybrid Intel, so the PCI scan keeps kms — but it is a real invariant, so I documented it in the conf (5d5cc06) rather than restructure the packaging. (That MODULES= clobber is arguably a pre-existing bug worth its own fix: it erases nvidia early-loading on such machines regardless of this PR.)
  2. .pacnew merged later gets no rebuild: at migration time the active config still carries kms, so the built image matches its config; whoever hand-merges a mkinitcpio .pacnew rebuilds per standard practice, and the next kernel update covers it regardless. Left as is.
  3. Marker race between two concurrent users: same accepted check/build/mark pattern as migrations/1786482992.sh; worst case is a redundant rebuild. Left as is.

./test/shell: all pass except the pre-existing, unrelated config-test.sh failure (sibling omarchy-pkgs PKGBUILD coverage for omarchy-crash-watch.service, untouched by this PR).

— 🤖 Claude, posting on behalf of @dhh

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (1)

migrations/1786605598.sh:1

  • The PR description says this update needs “no migration,” but this file adds a machine-wide migration specifically because the settings update does not rebuild the initramfs. Please update the PR description to reflect that existing Limine installs are rebuilt once by this migration; otherwise the stated rollout behavior contradicts the implementation.
echo "Rebuild the initramfs so NVIDIA-only systems shed nouveau's unused GSP firmware"

@dhh

dhh commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Please test the updated implementation 🙏

@dhh

dhh commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Tested the updated implementation at 5d5cc06 (includes matjam's b1058de + 94ed33e and the two fixup commits).

Test runs

  • test/shell.d/nvidia-kms-hook-test.sh — 9/9 pass, including the new inconclusive cases (empty tree, unreadable device alone, unreadable device beside a readable NVIDIA GPU).
  • test/shell.d/nvidia-kms-migration-test.sh — 6/6 pass; the migration sources the installed conf, so it inherits the inconclusive logic with no gating change needed.
  • Full ./test/shell — all 167 test files pass.

Direct simulation of the conditional against fabricated sysfs trees, all behaving as intended:

layout kms
NVIDIA-only (VGA 0x0300) dropped
dual NVIDIA GPUs dropped
NVIDIA GPU + readable non-display device dropped
Intel iGPU + NVIDIA 3D (0x0302) kept
AMD iGPU + NVIDIA 3D kept
empty PCI tree kept
NVIDIA audio function only kept
unreadable device beside NVIDIA GPU kept

The third row is the important regression check on 94ed33e: the inconclusive path only fires when class/vendor are actually unreadable — which enumerated sysfs devices never are — so a real NVIDIA-only machine full of readable chipset devices still gets the optimization, while anything genuinely unverifiable fails safe to keeping kms. A focused codex (xhigh) pass over 94ed33e and its composition with the migration returned no findings.

Verdict: sound. Nothing further to fix from my side.

— 🤖 Claude, posting on behalf of @dhh

nick1udwig pushed a commit to nick1udwig/omarchy that referenced this pull request Sep 9, 2026
omacom#6791)

* Drop the kms hook when the proprietary NVIDIA driver handles early KMS

install/hardware/nvidia.sh early-loads nvidia_drm (modeset=1) for early KMS,
but HOOKS still carried the kms hook, so autodetect pulled nouveau and
~100 MB of its GSP firmware into every initramfs for a driver that never
runs. On a Limine UKI setup that meant a 256 MB image where ~144 MB is
normal, doubled again by the fallback history on /boot.

Filter kms out of HOOKS when nvidia_drm is in MODULES (nvidia.conf sorts
before this drop-in) and every PCI display controller is NVIDIA. Hybrid
systems keep kms so the iGPU retains early KMS at the LUKS prompt.

Verified on an RTX 4090 (nvidia-open-dkms 610.57.04): UKI shrinks
256,183,296 -> 144,066,048 bytes, nouveau and its firmware gone, the
nvidia-utils GSP blobs and all four nvidia modules retained, Plymouth
still owns the LUKS prompt via nvidia_drm.

Fixes omacom#6790

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Address review: quote literals, harden PCI detection, add shell tests

Quote fixed string literals in the [[ ]] comparisons per AGENTS.md, and
read the PCI tree through OMARCHY_PCI_DEVICES_PATH, the same seam
bin/omarchy-hw-nvidia already uses.

Require a positively identified NVIDIA display controller before dropping
kms: an empty or unreadable PCI tree previously counted as "no non-NVIDIA
GPU" and would have dropped the hook. Unexpected trees now keep kms.

Cover the conditional in test/shell.d/nvidia-kms-hook-test.sh: nvidia-only,
hybrid, no nvidia_drm, MODULES unset under set -u, audio-function-only,
empty tree, and a device directory missing its sysfs attributes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Treat unreadable PCI devices as inconclusive and make the test hermetic

A device whose class/vendor attributes cannot be read could be another
GPU, so skipping it let a readable NVIDIA GPU beside it drop kms without
having verified the whole tree. Count it as a non-NVIDIA sighting so kms
stays, and cover the mixed case in the test.

The test also sourced the host's /etc/vconsole.conf under set -u, where a
valid KEYMAP-only file makes the XKBLAYOUT expansion fail in the subshell
and ties the result to the machine running it. Predefine XKBLAYOUT and
FILES before sourcing the config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Rebuild existing initramfses once the kms hook no longer applies

The settings package deploys the new omarchy_hooks.conf conditional, but
nothing rebuilds the initramfs when only a mkinitcpio drop-in changes, so
existing NVIDIA-only installs would carry nouveau's ~100 MB of GSP
firmware until their next kernel update. Following the precedent of
1784476564, add a migration that rebuilds via limine-mkinitcpio — once
per machine, and only where evaluating the installed drop-ins shows the
conditional actually dropped kms, so hybrid machines, non-NVIDIA
machines, and user-edited configs are left alone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Document the mid-sourcing MODULES caveat in the kms conditional

A later-sorting drop-in that resets MODULES outright (as
surface_device_modules.conf does) would strip nvidia_drm after kms was
already dropped. Every machine Omarchy writes such a file for is hybrid
Intel and keeps kms through the PCI scan, but that is worth stating so
the invariant is not broken by accident.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: David Heinemeier Hansson <david@hey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NVIDIA installs: kms hook bundles ~100 MB of unused nouveau GSP firmware into every initramfs/UKI

3 participants