A server that changes underneath its own running processes is the root cause of a surprising share of outages: a half-applied package update, a library swapped while a daemon still had the old one mapped, a config file edited by hand and never recorded anywhere. openSUSE Leap Immutable, the system most people still know as Leap Micro, removes that whole class of failure by refusing to modify the running root filesystem at all. Every change is built into a new btrfs snapshot first, and the machine only switches to it on the next boot.
This explainer opens a real system and shows the mechanism piece by piece: the read-only root, the btrfs subvolume layout, what transactional-update does when you install a package, how /etc is handled, and what a rollback restores (and what it quietly does not). It is the first part of our openSUSE Immutable series, which goes on to installation, Ignition and Combustion provisioning, Podman Quadlet workloads and a K3s edge cluster. The behavior described here was checked on Leap 16.1 in Immutable mode and on Leap Micro 6.2 in October 2026, and the two are compared side by side further down.
What happened to Leap Micro
Leap Micro has been folded into Leap itself. The openSUSE project announced in its Leap 16.1 release candidate announcement that Leap 16.1 in Immutable mode replaces Leap Micro, and that there will be no Leap Micro 6.3 or 7.0. Leap Micro 6.2 is therefore the last release of the old product line, and the openSUSE roadmap keeps it supported until after Leap 16.2 ships, which is expected around October 2027.
The change is mostly about packaging rather than technology. Leap Micro was a separate product with its own repositories and its own images, while Leap Immutable is a mode you pick in the same Agama installer that installs regular Leap, or an appliance image downloaded from the same Leap 16.1 tree. The machinery that makes the system immutable (btrfs snapshots, transactional-update, health-checker, Ignition and Combustion for first boot) carries over unchanged. Even /etc/os-release on a fresh Leap 16.1 Immutable host still identifies the variant as Micro:
grep -E "^(PRETTY_NAME|VARIANT|VARIANT_ID|ID_LIKE)=" /etc/os-releaseThe VARIANT_ID of transactional is the value scripts should test for when they need to detect an immutable openSUSE host, because the marketing name has already changed once and the variant ID has not:
PRETTY_NAME="openSUSE Leap 16.1"
VARIANT="Micro"
VARIANT_ID="transactional"
ID_LIKE="suse opensuse opensuse-microos"Everything in the rest of this article applies to both products, and where they differ we say so explicitly. If you are still choosing between the openSUSE family members, the Leap vs Tumbleweed vs SLES comparison covers the traditional editions.
What “immutable” means on openSUSE
On openSUSE, immutable means one specific thing: the root filesystem that holds /usr, the kernel modules and every installed package is mounted read-only while the system runs. It does not mean the machine cannot be changed, and it does not mean configuration is frozen. The quickest way to see the boundary is to look at the mount table and then try to write somewhere you normally could:
Three details in that output carry the whole design. The root is not the top of the btrfs filesystem but a specific snapshot, @/.snapshots/2/snapshot, and it carries the ro flag. /etc is a separate subvolume nested inside that same snapshot, mounted read-write. /var lives in its own subvolume entirely outside the snapshot tree. Even root cannot create a file under /usr/bin, because the refusal comes from the filesystem mount rather than from permissions, so sudo makes no difference.
How the btrfs subvolume layout divides the disk
Listing the subvolumes shows how the disk is split between things that belong to a snapshot and things that every snapshot shares. The output below is from the same Leap 16.1 Immutable VM, taken right after first-boot provisioning:
sudo btrfs subvolume list /Snapshot 1 is the image as it was built, snapshot 2 is the state after first-boot provisioning, and each of them owns its own etc child subvolume:
ID 256 gen 11 top level 5 path @
ID 257 gen 33 top level 256 path @/.snapshots
ID 258 gen 18 top level 257 path @/.snapshots/1/snapshot
ID 259 gen 42 top level 256 path @/home
ID 260 gen 31 top level 256 path @/opt
ID 261 gen 35 top level 256 path @/root
ID 262 gen 31 top level 256 path @/srv
ID 263 gen 42 top level 256 path @/var
ID 264 gen 31 top level 256 path @/boot/writable
ID 265 gen 31 top level 256 path @/usr/local
ID 266 gen 31 top level 256 path @/boot/grub2/i386-pc
ID 267 gen 31 top level 256 path @/boot/grub2/x86_64-efi
ID 268 gen 26 top level 258 path @/.snapshots/1/snapshot/etc
ID 269 gen 29 top level 257 path @/.snapshots/2/snapshot
ID 270 gen 42 top level 269 path etcReading the list as a map of where data survives a rollback gives a table worth keeping in mind before you put anything on one of these hosts:
| Path | Writable at runtime | Part of the snapshot | What belongs there |
|---|---|---|---|
/ (incl. /usr, /boot kernels) | No | Yes | Packages, libraries, kernel. Changed only by transactional-update |
/etc | Yes | Yes (own subvolume per snapshot) | Host configuration |
/var | Yes | No | Container storage, logs, databases, package caches |
/home, /root | Yes | No | User data, SSH keys |
/opt, /srv, /usr/local | Yes | No | Locally built tools and data you place by hand |
/boot/writable | Yes | No | Bootloader state such as the first-boot marker |
The key insight is that the snapshot covers the operating system and its configuration, while everything a workload produces lives outside it. That is what makes it safe to roll the OS back to yesterday without losing a database that was written to all night. It also means a rollback is not a backup of your data, a point we return to at the end. The layout itself is standard btrfs; the btrfs subvolumes and snapshots primer explains the primitives if they are new to you.
What transactional-update actually does during an update
Because zypper cannot write to a read-only root, every package operation goes through transactional-update, a wrapper around the lower-level tukit tool. What’s actually happening when you run it is a short, fixed sequence. It takes a btrfs snapshot of the currently running root, mounts that new snapshot read-write somewhere temporary, runs zypper with -R pointed at that mount so the packages land inside the copy, and finally marks the copy as the default subvolume for the next boot. The running system is never touched.
Installing a single small package is enough to watch every stage. We installed tmux, then immediately tried to run it and listed the snapshots:
sudo transactional-update -n pkg install tmuxThe trimmed output below keeps the lines that matter. Note that tmux is not available after the command finishes, and that snapper now shows two different markers:
In snapper list, the minus sign marks the snapshot that is currently mounted as root and the plus sign marks the one that will be booted next. A snapshot that is both running and default shows an asterisk instead, which is what you see on a host with no pending changes. This two-marker state is the one to recognise in production: a host showing 2- and 3+ has an update staged and is still running the old code, so anything you verify on it right now is verifying the old system.
Two further details from the full log are worth understanding. The first transaction on a new host printed several SELinux: Relabeled /var lines, because SELinux is enforcing by default and tukit fixes the labels on the shared /var tree before using it. Creating the snapshot itself is close to instant on btrfs, because it is a copy-on-write reference rather than a copy of the data; the real cost of an update is the download and the reboot.
After a reboot, the default snapshot becomes the running one and the package exists:
findmnt -n -o SOURCE /; tmux -VThe root now comes from snapshot 3:
/dev/sda3[/@/.snapshots/3/snapshot]
tmux 3.5aOn our two-vCPU VM, SSH was answering again within about 11 seconds of issuing the reboot, and systemd-analyze reported a 6.3 second boot. On these hosts a reboot is a routine part of every change rather than an event, which is why they are usually paired with a maintenance window and an orchestrator that drains workloads first. Day-to-day usage of every subcommand, including shell, apply and run, gets its own article later in the series.
How /etc changes are carried between snapshots
Configuration is the awkward part of any snapshot design, because people edit /etc on the running system at any moment, including after an update has already been staged. On current Leap Immutable and Leap Micro, /etc is a btrfs subvolume nested inside each root snapshot (older MicroOS releases used an overlayfs here, which is why one of the health-checker plugins is still named etc-overlayfs.sh). When transactional-update creates snapshot 3, it copies the current /etc into it, and at the next boot it synchronises any changes made to the running /etc in the meantime.
We tested the timing that catches people out. After snapshot 3 was staged, but before rebooting, we changed the time zone on the running system:
sudo timedatectl set-timezone Africa/NairobiAfter booting into snapshot 3, /etc/localtime pointed at Africa/Nairobi and snapper started printing its dates in EAT, so the late edit was carried forward. In a separate test on another VM, an appended line in an existing file and a brand new file created under /etc after staging both survived the reboot as well. The practical rule is simple: on these releases you can keep editing /etc between staging an update and rebooting, and the edits follow you into the new snapshot. Files that a package update also changes are the exception, because the package version and your edit then have to be reconciled, and that conflict case is covered in the transactional-update part of the series.
A second source of confusion is that some defaults no longer live in /etc at all. On our host, /etc/transactional-update.conf did not exist; the shipped default sat in /usr/etc/transactional-update.conf and contained a single line, UPDATE_METHOD=dup. Vendor defaults in /usr/etc are read-only and change with packages, while a file you create in /etc overrides them. If you go looking for a config file and find nothing in /etc, check /usr/etc before assuming the setting is unconfigured.
What a rollback restores and what it keeps
A rollback on an immutable openSUSE host does not undo anything in place. It changes which snapshot is the default and asks you to reboot, which is exactly the mirror image of an update. We rolled back to snapshot 2, rebooted, and checked the package and the time zone:
The package is gone, as expected, because /usr now comes from snapshot 2 again. The time zone change survived. This matters because the timezone edit was made while snapshot 2 was running, so it was written into snapshot 2’s own /etc subvolume before being copied forward into snapshot 3. Rolling back to snapshot 2 therefore brings the edit with it. A rollback reverts the operating system to the state of the target snapshot’s /etc, which includes every live edit made while that snapshot was the one running.
Snapper also records the rollback in a way that protects you. Before the rollback, snapshot 2 had the cleanup algorithm number, which means the daily snapper-cleanup.timer may delete it once the count exceeds the limit (the default NUMBER_LIMIT is 2-10). After the rollback, the cleanup column for snapshot 2 was empty, so the snapshot you rolled back to is no longer eligible for automatic deletion. The abandoned snapshot 3 keeps its number flag and ages out normally.
The same snapper mechanics exist on regular openSUSE and Fedora installs with a btrfs root, where they wrap ordinary zypper or dnf transactions; the Snapper automatic rollback guide shows that model. The difference on Leap Immutable is that the snapshot is the only way to change the system, so there is never a moment where the running root is half updated.
The services that update and protect the host unattended
On a fresh install, four units cooperate so that a host can update itself without anyone logging in. We checked their state and schedules directly on the VM:
systemctl cat transactional-update.timer | grep -E "^(OnCalendar|RandomizedDelaySec|Persistent)"; rebootmgrctl get-strategy; rebootmgrctl get-windowThe update timer fires daily with up to two hours of random delay, and reboots are deferred to a nightly window:
OnCalendar=daily
RandomizedDelaySec=2h
Persistent=true
Reboot strategy: best-effort
Maintenance window is set to '*-*-* 03:30:00', lasting 01:30.Put together, the mechanism is a pipeline. transactional-update.timer runs the configured update method into a new snapshot. If anything changed, it asks rebootmgr for a reboot, and with the best-effort strategy rebootmgr waits for the 03:30 window (lasting 90 minutes) instead of rebooting immediately. After the reboot, health-checker runs a set of plugin checks found in /usr/libexec/health-checker/: btrfs subvolumes mounted, /etc mounted correctly, the RPM database consistent, logind and /tmp healthy, and rebootmgr reachable. If the new snapshot fails those checks, health-checker marks it bad and reboots into the last snapshot that booted successfully. Finally, snapper-cleanup.timer prunes old snapshots according to the number limits.
One consequence surprises people who come from traditional servers: an untouched Leap Immutable host will reboot itself in the early morning whenever an update lands. That is the intended behavior for an edge box or a container host. For a node in a cluster, you either coordinate the reboots through the orchestrator or change the strategy, both of which are shown later in the series.
How Leap 16.1 Immutable differs from Leap Micro 6.2
We provisioned both systems from their official qcow2 appliance images with the same Combustion script (minus the wheel membership on 16.1) and compared them on identical VMs. The architecture is the same, including the per-snapshot /etc subvolume and the identical subvolume list. The differences are in the defaults and the package versions, and two of them break existing automation:
| Item | Leap 16.1 Immutable | Leap Micro 6.2 |
|---|---|---|
Daily update UPDATE_METHOD | dup | up |
wheel group in the base image | Absent | Present (gid 497) |
| Repositories | Leap 16.1 repo-oss, repo-non-oss, openh264 | repo-main from the leap-micro/6.2 tree |
| transactional-update | 6.1.1 | 5.0.7 |
| Kernel | 6.12.0-160100.7.2 | 6.12.0-160000.35.1 |
| Podman | 5.8.2 | 5.4.2 |
| Ignition | 2.26.0 | 2.21.0 |
| Installed packages (Base image) | 503 | 535 |
| SELinux | Enforcing, targeted | Enforcing, targeted |
| Reboot window | 03:30, 1h30 | 03:30, 1h30 |
The missing wheel group is the one we hit first. Our Combustion script created the admin user with useradd -G wheel, which works on Leap Micro 6.2. On Leap 16.1 Immutable the first boot failed with useradd: group 'wheel' does not exist, Combustion rolled its snapshot back, and the VM stopped in the emergency shell. Creating the group would not help on its own. In both releases’ appliance images the %wheel lines in /usr/etc/sudoers are commented out and /usr/etc/sudoers.d/ is empty, so wheel membership never granted sudo by itself, even on Leap Micro. (An Agama install in Immutable mode is different: it creates wheel and ships a rule for it.) Any first-boot script carried over from Leap Micro should grant sudo through a file in /etc/sudoers.d/ named after the specific user. The switch from up to dup matters as well: unattended updates on Leap 16.1 Immutable can now downgrade, replace (rename) or remove packages the way a distribution upgrade does, instead of only updating what is already installed.
How it compares to MicroOS, Fedora CoreOS, Flatcar and Talos
Every immutable server OS solves the same problem, but they disagree on how the new system image is produced and how much of a normal Linux userland you keep. The comparison that matters for choosing one is the update model and the management interface:
| System | Base | How updates are applied | First-boot config | Shell and packages |
|---|---|---|---|---|
| Leap Immutable / Leap Micro | Leap (SLE sources), fixed release | btrfs snapshot + zypper into it, reboot | Ignition, Combustion, jeos-firstboot | Full shell, RPMs via transactional-update |
| openSUSE MicroOS | Tumbleweed, rolling | Same as above | Same as above | Same as above |
| Fedora CoreOS | Fedora, rolling streams | rpm-ostree image deployments, auto-updated by Zincati | Ignition (written as Butane) | Full shell, layering discouraged |
| Flatcar Container Linux | Gentoo-derived, own releases | A/B /usr partitions via update_engine | Ignition (Butane) | Full shell, no package manager |
| Talos Linux | Purpose-built for Kubernetes | A/B image upgrades through the API | Machine config YAML | No shell or SSH, managed with talosctl |
The openSUSE approach sits at the conventional end of this range. Because the new snapshot is built by running real zypper transactions, you can still install an arbitrary RPM onto the host when you genuinely need one, such as a storage driver or a monitoring agent, and the result is atomic and reversible. Image-based systems like Fedora CoreOS and Flatcar give stronger guarantees that every host is byte-identical, at the price of making host-level packages awkward. Talos goes furthest by removing the shell entirely. We benchmarked several of these as Kubernetes hosts in the minimal Kubernetes OS comparison, which is worth reading if a cluster is your end goal.
When an immutable openSUSE host is the right choice
Leap Immutable fits machines whose job is to run workloads rather than be administered: container hosts, K3s nodes, edge boxes at remote sites, virtualization hosts for a few VMs, and appliances that must update themselves without a person on call. In all of those cases the workload lives in containers or VMs under /var, the host changes rarely, and the ability to roll back a bad update with one command and one reboot is worth far more than the ability to install packages casually. The next part of this series covers the installation paths for exactly these roles, from a USB self-install image to a qcow2 appliance on KVM.
It fits poorly where the host itself is the workload and changes constantly. A developer workstation that installs build dependencies several times a day, a server where an application vendor’s installer writes into /usr or /opt and expects to be run interactively, or a host managed by configuration tooling that assumes it can install packages and restart services in one pass will all fight the reboot-per-change model. Regular Leap with snapper, managed through Cockpit or zypper, is the better choice there, and the zypper cheat sheet covers its day-to-day commands. Developer tooling on an immutable host belongs in containers, which is the problem Distrobox and Toolbox were built to solve.
Common misconceptions about immutable openSUSE
“Immutable means you cannot install packages.” You can install any package from the repositories with transactional-update pkg install. The constraint is that the package appears only after a reboot, and that it arrives in a new snapshot you can discard. The recommended practice is to keep host packages to the minimum the hardware and the container runtime need, not to avoid them entirely.
“A rollback undoes my configuration changes.” Only partially, as the time zone test showed. A rollback restores the /etc of the target snapshot, and that subvolume already contains every edit made while the target was the running system. If you need to undo a config change, revert the file itself or keep /etc under version control; do not rely on a rollback to do it.
“Snapshots are my backup.” They are not. Snapshots live on the same disk as the data they protect, they exclude /var, /home and /srv by design, and snapper deletes old ones automatically. Container volumes and databases on an immutable host need the same off-host backups as on any other server.
“Leap Micro is dead, so this knowledge is obsolete.” The product name is retired, but the system is not. Leap 16.1 Immutable runs the same transactional-update, the same snapshot layout and the same first-boot tooling, and Leap Micro 6.2 installations can be moved into it with openSUSE’s migration tool, which the project still labels experimental. Everything you learn on one applies directly to the other, and to MicroOS and SUSE Linux Micro as well.
“The read-only root makes the host secure.” It makes a class of tampering and drift harder, because nothing running on the system can silently replace a binary in /usr. It does nothing for a vulnerable service exposed on the network, weak SSH settings or an unencrypted disk, which is why hardening with TPM2 disk encryption, SELinux and SSH lockdown has its own part in this series.
