Repository navigation
runtime: steal tasks from the LIFO slot - #7431
Merged
Merged
Conversation
This commit changes the LIFO slot on multi-threaded runtime workers from being a mutable `Option<Notified<Arc<Handle>>>` to a new `AtomicNotified` type. This type implements a cell containing a nullable task pointer which can be swapped atomically. It's analogous to `AtomicCell` but with the extra `PhantomData` to remember the task's scheduler type parameter, which would otherwise be erased by the conversion into a `*mut Header`` pointer. This change is in preparation for a subsequent change to allow work-stealing from the LIFO slot (see: #4941).
This way, it's accesible by the stealer. Leave all the LIFO *accounting* (i.e. deciding whether we hit the LIFO slot or not) up to the worker. Gotta figure out whether the load of lifo presence will race...ugh.
This commit adds a test ensuring that if a task is notified to the LIFO slot by another task which then blocks the worker thread forever, the LIFO task is eventually stolen by another worker. I've confimed that this test fails on the `master` branch, and passes after these changes.
This test spawns a task that sometimes ends up in the LIFO slot and sometimes doesn't. This was previously fine as the LIFO slot didn't count for `worker_local_queue_depth`, but now it does. Thus, we have to make sure that task no longer exists before asserting about queue depth.
hawkw
marked this pull request as ready for review
July 1, 2025 19:39
Member
Author
|
(shoutout to @mkeeter for nerd-sniping me into actually doing this) |
ADD-SP
reviewed
Jul 2, 2025
Comment on lines
+463
to
+465
| // If we also grabbed the task from the LIFO slot, include that in the | ||
| // steal count as well. | ||
| dst_stats.incr_steal_count(n as u16 + lifo.is_some() as u16); |
Member
There was a problem hiding this comment.
It's also possible to use u16::from(bool), if you care about cast_lossless cleanliness.
hawkw
added a commit
to oxidecomputer/omicron
that referenced
this pull request
Apr 15, 2026
This branch updates our dependency on `oxide-tokio-rt` to pick up Tokio v1.52.0 and the corresponding changes to the default runtime settings in `oxide-tokio-rt`. In particular, this allows us to pick up two of my upstream fixes in Tokio for a pair of issues that have been major thorns in our side for some time: * Tokio PR tokio-rs/tokio#7431, released in [Tokio v1.51.0], changes the multi-threaded runtime to allow tasks in the LIFO slot to participate in work-stealing. Therefore, it should no longer be necessary to disable the LIFO slot optimization, as the pathology described in #8334, where a task placed in the LIFO slot can become permanently or semi-permanently stuck while the task that notified them runs for a long time without yielding, can no longer occur. `oxide-tokio-rt` v0.1.4 removes the runtime configuration to disable the LIFO slot as the issue has been fixed upstream. * Tokio PR tokio-rs/tokio#8010, released in [Tokio v1.52.0], which adds eager handoff for the I/O and time drivers in the multi-threaded runtime. This is currently an experimental feature, although it is your author's opinion that this is really a fix for incorrect runtime behavior.[^1] It changes worker threads in the multi-threaded runtime to wake another worker prior to polling tasks if that worker had previously been parked on the I/O driver or timer wheel. Eagerly handing off these resources should prevent pathologies such as #9619. `oxide-tokio-rt` v0.1.4 enables this behavior by default. Fixes #8334 Fixes #9619 [Tokio v1.52.0]: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.0 [Tokio v1.51.0]: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.51.0 [^1]: Cue @leftwo's dictum that a "performance regression" from fixing incorrect behavior...isn't a performance regression.
hawkw
added a commit
to oxidecomputer/omicron
that referenced
this pull request
Apr 16, 2026
Commit #10272 updated our dependency on `oxide-tokio-rt` to v0.1.4 and our `tokio` dependency to v1.52.0. This allowed us to pick up two of my fixes for Tokio issues that have been a thorn in our side for a long time, tokio-rs/tokio#7431 and tokio-rs/tokio#8010, which fix #8334 and #9619, respectively. The nature of these fixes is described in greater detail in #10272. Unfortunately, #10272 had to be reverted (in #10279), since @iliana discovered an unrelated regression in Tokio v1.52.0, tokio-rs/tokio#8056 (our issue #10277). This regression caused `spawn_blocking` to occasionally hang, and was introduced in tokio-rs/tokio@1604bc3 (PR tokio-rs/tokio#7757). I've since reverted this change upstream (tokio-rs/tokio#8057), and published a patch release ([v1.52.1]), which fixes the regression. Therefore, it is now once again safe to update our Tokio dependency to pick up the other fixes. This commit does that. I've also confirmed that the issue described in #10277 is no longer present in Tokio v1.52.1, as demonstrated by the fact `cargo nextest run -p omicron-sled-agent --stress-count 100 -- --exact artifact_store::test::issue_7796` now succeeds without hanging once again. Fixes #10272
hawkw
added a commit
that referenced
this pull request
Apr 20, 2026
Currently, the `rt_threaded::lifo_stealable` test I added in #7431 spawns an additional task which sleeps on a 4ms timer in a loop. This ensures that no worker remains permanently parked. This was added because it was necessary to stop the LIFO slot deadlock from occurring prior to changes in the logic for determining whether to notify another worker, which is what @Darksonn was referring to in [this comment][1]. Removing the `churn()` test makes the test actually validate that another worker is notified to steal the LIFO task, and that the changes from #7431 will *always* prevent a LIFO slot deadlock, regardless of the behavior of other tasks on the runtime. See also [this comment][2] for further discussion. [1]: #7431 (comment) [2]: #8069 (comment)
Darksonn
added a commit
to Darksonn/tokio
that referenced
this pull request
May 1, 2026
This reverts commit eeb55c7.
Darksonn
added a commit
to Darksonn/tokio
that referenced
this pull request
May 2, 2026
This reverts commit eeb55c7.
Darksonn
added a commit
to Darksonn/tokio
that referenced
this pull request
May 2, 2026
This reverts commit eeb55c7.
Darksonn
added a commit
to Darksonn/tokio
that referenced
this pull request
May 2, 2026
This reverts commit eeb55c7.
This was referenced May 4, 2026
eleboucher
pushed a commit
to eleboucher/towonel
that referenced
this pull request
May 4, 2026
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | workspace.dependencies | patch | `1.52.1` → `1.52.2` | --- ### Release Notes <details> <summary>tokio-rs/tokio (tokio)</summary> ### [`v1.52.2`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.2): Tokio v1.52.2 [Compare Source](tokio-rs/tokio@tokio-1.52.1...tokio-1.52.2) ### 1.52.2 (May 4th, 2026) This release reverts the LIFO slot stealing change introduced in 1.51.0 ([#​7431]), due to [its performance impact][#​8065]. ([#​8100]) [#​7431]: tokio-rs/tokio#7431 [#​8065]: tokio-rs/tokio#8065 [#​8100]: tokio-rs/tokio#8100 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL3BhdGNoIl19--> Reviewed-on: https://git.erwanleboucher.dev/eleboucher/towonel/pulls/31
1 task
hawkw
added a commit
to oxidecomputer/oxide-tokio-rt
that referenced
this pull request
May 4, 2026
This reverts commit 66c189b, which removed the code for disabling Tokio's LIFO slot optimization. We re-enabled the LIFO slot since tokio-rs/tokio#7431 allowed it to participate in work-stealing, and this change was released in Tokio v1.51.0. However, Tokio v1.52.2 contained commit tokio-rs/tokio#8100, which reverted tokio-rs/tokio#7431, due to a performance regression in some applications (tokio-rs/tokio#8065). Therefore, this means that if we update to Tokio v1.52.2, we will once again be susceptible to LIFO-slot-related deadlocks. This commit re-disables the LIFO slot. Hopefully, we will be able to figure out a solution upstream that allows stealing LIFO tasks while reducing spurious cross-thread wakeups. But for now, I think that we are best off leaving this disabled. Sigh.
hawkw
added a commit
to oxidecomputer/oxide-tokio-rt
that referenced
this pull request
May 4, 2026
This reverts commit 66c189b, which removed the code for disabling Tokio's LIFO slot optimization. We re-enabled the LIFO slot since tokio-rs/tokio#7431 allowed it to participate in work-stealing, and this change was released in Tokio v1.51.0. However, Tokio v1.52.2 contained commit tokio-rs/tokio#8100, which reverted tokio-rs/tokio#7431, due to a performance regression in some applications (tokio-rs/tokio#8065). Therefore, this means that if we update to Tokio v1.52.2, we will once again be susceptible to LIFO-slot-related deadlocks. This commit re-disables the LIFO slot. Hopefully, we will be able to figure out a solution upstream that allows stealing LIFO tasks while reducing spurious cross-thread wakeups. But for now, I think that we are best off leaving this disabled. Sigh.
hawkw
added a commit
that referenced
this pull request
May 5, 2026
This commit puts back the change to allow stealing tasks from the LIFO slot, which was originally introduced in #7431. This change was reverted in 9605999 in order to fix the performance regression due to increased cross thread wakeups (#8065). Previously, pushing a task to an unoccupied LIFO slot would not notify a parked worker, but when #7431 made the LIFO slot stealable, we started notifying in that case, which increases runtime overhead. Therefore, the change was reverted in v1.52.2. This commit reintroduces it. Subsequent commits on this branch will allow configuring whether LIFO slot pushes wake a parked worker, so that we can allow stealing from the LIFO slot while making the increased wakeup overhead opt-in. Closes #4941
hawkw
added a commit
that referenced
this pull request
May 5, 2026
In #7431, I originally implemented `AtomicNotified` with the `AcqRel` atomic ordering, which is all that *should* be necessary here. While trying to debug a failing test on ARM, I changed it to `SeqCst`. This fixed the test, but was not actually necessary to solve the root cause of that test failure, which was ultimately fixed in #8008 instead. Using `SeqCst` here and locking the bus probably increases the overhead of using the LIFO slot substantially, so I've un-done that.
hawkw
added a commit
that referenced
this pull request
May 5, 2026
Currently, the `rt_threaded::lifo_stealable` test I added in #7431 spawns an additional task which sleeps on a 4ms timer in a loop. This ensures that no worker remains permanently parked. This was added because it was necessary to stop the LIFO slot deadlock from occurring prior to changes in the logic for determining whether to notify another worker, which is what @Darksonn was referring to in [this comment][1]. Removing the `churn()` test makes the test actually validate that another worker is notified to steal the LIFO task, and that the changes from #7431 will *always* prevent a LIFO slot deadlock, regardless of the behavior of other tasks on the runtime. See also [this comment][2] for further discussion. [1]: #7431 (comment) [2]: #8069 (comment)
eleboucher
pushed a commit
to eleboucher/towonel
that referenced
this pull request
May 20, 2026
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [tokio](https://tokio.rs) ([source](https://github.com/tokio-rs/tokio)) | workspace.dependencies | patch | `1.52.1` → `1.52.2` | --- ### Release Notes <details> <summary>tokio-rs/tokio (tokio)</summary> ### [`v1.52.2`](https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.2): Tokio v1.52.2 [Compare Source](tokio-rs/tokio@tokio-1.52.1...tokio-1.52.2) ### 1.52.2 (May 4th, 2026) This release reverts the LIFO slot stealing change introduced in 1.51.0 ([#​7431]), due to [its performance impact][#​8065]. ([#​8100]) [#​7431]: tokio-rs/tokio#7431 [#​8065]: tokio-rs/tokio#8065 [#​8100]: tokio-rs/tokio#8100 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMDEuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEwMS4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJ0eXBlL3BhdGNoIl19--> Reviewed-on: https://git.erwanleboucher.dev/eleboucher/towonel/pulls/31
6 tasks
ahacop
added a commit
to ahacop/erwindb
that referenced
this pull request
Sep 28, 2026
[cb skip] Changes in 1.53.1 ----------------- 1.53.1 (July 20th, 2026) ------------------------ Fixed ----- - signal: restore MSRV by removing `OnceLock::wait` from the Windows handler ([#8300]) Fixed (unstable) ---------------- - time: fix alt timer cancellation and insertion race ([#8252]) Documented ---------- - runtime: remove dead link definition in Runtime::block_on ([#8301]) [#8252]: tokio-rs/tokio#8252 [#8300]: tokio-rs/tokio#8300 [#8301]: tokio-rs/tokio#8301 Changes in 1.53.0 ----------------- 1.53.0 (July 17th, 2026) ------------------------ Added ----- - fs: implement `From<OwnedFd>` and `From<OwnedHandle>` for `File` ([#8266]) - metrics: add task schedule latency metric ([#7986]) - net: add `SocketAddr` methods to Unix sockets ([#8144]) Changed ------- - io: add `#[inline]` to IO trait impls for in-memory types ([#8242]) - net: implement UCred::pid on FreeBSD ([#8086]) - net: support Nuttx target os ([#8259]) - signal: refactor global variables on Windows ([#8231]) - sync: `mpsc::{Receiver,UnboundedReceiver}` now drops waker on drop, even if there are still senders ([#8095]) - taskdump: support taskdumps on s390x ([#8192]) - time: add `#[track_caller]` to `timeout_at()` ([#8077]) - time: consolidate mutex locks on spurious poll ([#8124]) - time: defer waker clone on spurious poll ([#8107]) - time: move lazy-registration state into `Sleep` ([#8132]) - tracing: remove unnecessary span clone ([#8126]) Fixed ----- - io: do not treat zero-length reads as EOF in `Chain` ([#8251]) - net: use getpeereid for QNX peer credentials ([#8270]) - runtime: avoid illegal state in `FastRand` ([#8078]) - sync: wake mpsc receiver when a queued `reserve[_many]` returns permits ([#8260]) - taskdump: skip double wake on `Trace::capture`/`Trace::trace_with` ([#8043]) - time: avoid stack overflow in runtime constructor ([#8093]) - time (alt timer): ensure timers stay in the same runtime after `.reset()` ([#8169]) IO uring (unstable) ------------------- - fs: use io-uring for `fs::try_exists` ([#8080]) - fs: use io-uring for renaming files ([#7800]) - rt: flush io-uring CQE in case of CQE overflow ([#8277]) Documented ---------- - docs: clarify cancel safety wording ([#8181]) - fs: clarify `create_dir_all` succeeds if path exists ([#8149]) - io: add warning about stdout reordering with multiple handles ([#8276]) - net: document pipe `try_read*`/`try_write*` readiness behavior ([#8032]) - runtime: document interaction with fork() ([#8202]) - sync: clarify broadcast lagging semantics ([#8239]) - sync: document memory ordering guarantees for Semaphore ([#8119]) - task: explain why `yield_now` defers its waker ([#8254]) - time: add panic docs to `timeout_at()` ([#8077]) - time: fix reversed poll order in timeout doc ([#8214]) [#7800]: tokio-rs/tokio#7800 [#7986]: tokio-rs/tokio#7986 [#8032]: tokio-rs/tokio#8032 [#8043]: tokio-rs/tokio#8043 [#8077]: tokio-rs/tokio#8077 [#8078]: tokio-rs/tokio#8078 [#8080]: tokio-rs/tokio#8080 [#8086]: tokio-rs/tokio#8086 [#8093]: tokio-rs/tokio#8093 [#8095]: tokio-rs/tokio#8095 [#8107]: tokio-rs/tokio#8107 [#8119]: tokio-rs/tokio#8119 [#8124]: tokio-rs/tokio#8124 [#8126]: tokio-rs/tokio#8126 [#8132]: tokio-rs/tokio#8132 [#8144]: tokio-rs/tokio#8144 [#8149]: tokio-rs/tokio#8149 [#8169]: tokio-rs/tokio#8169 [#8181]: tokio-rs/tokio#8181 [#8192]: tokio-rs/tokio#8192 [#8193]: tokio-rs/tokio#8193 [#8202]: tokio-rs/tokio#8202 [#8214]: tokio-rs/tokio#8214 [#8231]: tokio-rs/tokio#8231 [#8239]: tokio-rs/tokio#8239 [#8242]: tokio-rs/tokio#8242 [#8251]: tokio-rs/tokio#8251 [#8254]: tokio-rs/tokio#8254 [#8259]: tokio-rs/tokio#8259 [#8260]: tokio-rs/tokio#8260 [#8266]: tokio-rs/tokio#8266 [#8270]: tokio-rs/tokio#8270 [#8276]: tokio-rs/tokio#8276 [#8277]: tokio-rs/tokio#8277 Changes in 1.52.4 ----------------- 1.52.4 (July 16th, 2026) ------------------------ Fixed ----- - runtime: don't skip the driver when `before_park` schedules work ([#8222]) Fixed (unstable) ---------------- - taskdump: remove crate disambiguators from output ([#8264]) [#8264]: tokio-rs/tokio#8264 [#8222]: tokio-rs/tokio#8222 Changes in 1.52.3 ----------------- 1.52.3 (May 8th, 2026) ---------------------- Fixed ----- * sync: fix underflow in mpsc channel `len()` ([#8062]) * sync: notify receivers in mpsc `OwnedPermit::release()` method ([#8075]) * sync: require that an `RwLock` has `max_readers != 0` ([#8076]) * sync: return `Empty` from `try_recv()` when mpsc is closed with outstanding permits ([#8074]) [#8062]: tokio-rs/tokio#8062 [#8074]: tokio-rs/tokio#8074 [#8075]: tokio-rs/tokio#8075 [#8076]: tokio-rs/tokio#8076 Changes in 1.52.2 ----------------- 1.52.2 (May 4th, 2026) ---------------------- This release reverts the LIFO slot stealing change introduced in 1.51.0 ([#7431]), due to [its performance impact][#8065]. ([#8100]) [#7431]: tokio-rs/tokio#7431 [#8065]: tokio-rs/tokio#8065 [#8100]: tokio-rs/tokio#8100 Changes in 1.52.1 ----------------- 1.52.1 (April 16th, 2026) ------------------------- Fixed ----- - runtime: revert [#7757] to fix [a regression][#8056] that causes `spawn_blocking` to hang ([#8057]) [#7757]: tokio-rs/tokio#7757 [#8056]: tokio-rs/tokio#8056 [#8057]: tokio-rs/tokio#8057 Changes in 1.52.0 ----------------- 1.52.0 (April 14th, 2026) ------------------------- Added ----- - io: `AioSource::register_borrowed` for I/O safety support ([#7992]) - net: add `try_io` function to `unix::pipe` sender and receiver types ([#8030]) Added (unstable) ---------------- - runtime: `Builder::enable_eager_driver_handoff` setting enable eager hand off of the I/O and time drivers before polling tasks ([#8010]) - taskdump: add `trace_with()` for customized task dumps ([#8025]) - taskdump: allow `impl FnMut()` in `trace_with` instead of just `fn()` ([#8040]) - fs: support `io_uring` in `AsyncRead` for `File` ([#7907]) Changed ------- - runtime: improve `spawn_blocking` scalability with sharded queue ([#7757]) - runtime: use `compare_exchange_weak()` in worker queue ([#8028]) Fixed ----- - runtime: overflow second half of tasks when local queue is filled instead of first half ([#8029]) Documented ---------- - docs: fix typo in `oneshot::Sender::send` docs ([#8026]) - docs: hide #[tokio::main] attribute in the docs of `sync::watch` ([#8035]) - net: add docs on `ConnectionRefused` errors with UDP sockets ([#7870]) [#7757]: tokio-rs/tokio#7757 [#7870]: tokio-rs/tokio#7870 [#7907]: tokio-rs/tokio#7907 [#7992]: tokio-rs/tokio#7992 [#8010]: tokio-rs/tokio#8010 [#8025]: tokio-rs/tokio#8025 [#8026]: tokio-rs/tokio#8026 [#8028]: tokio-rs/tokio#8028 [#8029]: tokio-rs/tokio#8029 [#8030]: tokio-rs/tokio#8030 [#8035]: tokio-rs/tokio#8035 [#8040]: tokio-rs/tokio#8040 Changes in 1.51.4 ----------------- 1.51.4 (July 16th, 2026) ------------------------ Fixed ----- - runtime: don't skip the driver when `before_park` schedules work ([#8222]) [#8222]: tokio-rs/tokio#8222 Changes in 1.51.3 ----------------- 1.51.3 (May 8th, 2026) ---------------------- Fixed ----- * sync: fix underflow in mpsc channel `len()` ([#8062]) * sync: notify receivers in mpsc `OwnedPermit::release()` method ([#8075]) * sync: require that an `RwLock` has `max_readers != 0` ([#8076]) * sync: return `Empty` from `try_recv()` when mpsc is closed with outstanding permits ([#8074]) [#8062]: tokio-rs/tokio#8062 [#8074]: tokio-rs/tokio#8074 [#8075]: tokio-rs/tokio#8075 [#8076]: tokio-rs/tokio#8076 Changes in 1.51.2 ----------------- 1.51.2 (May 4th, 2026) ---------------------- This release reverts the LIFO slot stealing change introduced in 1.51.0 ([#7431]), due to [its performance impact][#8065]. ([#8100]) [#7431]: tokio-rs/tokio#7431 [#8065]: tokio-rs/tokio#8065 [#8100]: tokio-rs/tokio#8100 Changes in 1.51.1 ----------------- 1.51.1 (April 8th, 2026) ------------------------ Fixed ----- - sync: fix semaphore reopens after forget ([#8021]) - net: surface errors from `SO_ERROR` on `recv` for UDP sockets on Linux ([#8001]) Fixed (unstable) ---------------- - metrics: fix `worker_local_schedule_count` test ([#8008]) - rt: do not leak fd when cancelling io\_uring open operation ([#7983]) [#7983]: tokio-rs/tokio#7983 [#8001]: tokio-rs/tokio#8001 [#8008]: tokio-rs/tokio#8008 [#8021]: tokio-rs/tokio#8021 Changes in 1.51.0 ----------------- 1.51.0 (April 3rd, 2026) ------------------------ Added ----- - net: implement `get_peer_cred` on Hurd ([#7989]) - runtime: add `tokio::runtime::worker_index()` ([#7921]) - runtime: add runtime name ([#7924]) - runtime: stabilize `LocalRuntime` ([#7557]) - wasm: add wasm32-wasip2 networking support ([#7933]) Changed ------- - runtime: steal tasks from the LIFO slot ([#7431]) Fixed ----- - docs: do not show "Available on non-loom only." doc label ([#7977]) - macros: improve overall macro hygiene ([#7997]) - sync: fix `notify_waiters` priority in `Notify` ([#7996]) - sync: fix panic in `Chan::recv_many` when called with non-empty vector on closed channel ([#7991]) [#7431]: tokio-rs/tokio#7431 [#7557]: tokio-rs/tokio#7557 [#7921]: tokio-rs/tokio#7921 [#7924]: tokio-rs/tokio#7924 [#7933]: tokio-rs/tokio#7933 [#7977]: tokio-rs/tokio#7977 [#7989]: tokio-rs/tokio#7989 [#7991]: tokio-rs/tokio#7991 [#7996]: tokio-rs/tokio#7996 [#7997]: tokio-rs/tokio#7997 Changes in 1.50.0 ----------------- 1.50.0 (Mar 3rd, 2026) ---------------------- Added ----- - net: add `TcpStream::set_zero_linger` ([#7837]) - rt: add `is_rt_shutdown_err` ([#7771]) Changed ------- - io: add optimizer hint that `memchr` returns in-bounds pointer ([#7792]) - io: implement vectored writes for `write_buf` ([#7871]) - runtime: panic when `event_interval` is set to 0 ([#7838]) - runtime: shorten default thread name to fit in Linux limit ([#7880]) - signal: remember the result of `SetConsoleCtrlHandler` ([#7833]) - signal: specialize windows `Registry` ([#7885]) Fixed ----- - io: always cleanup `AsyncFd` registration list on deregister ([#7773]) - macros: remove (most) local `use` declarations in `tokio::select!` ([#7929]) - net: fix `GET_BUF_SIZE` constant for `target_os = "android"` ([#7889]) - runtime: avoid redundant unpark in current_thread scheduler ([#7834]) - runtime: don't park in `current_thread` if `before_park` defers waker ([#7835]) - io: fix write readiness on ESP32 on short writes ([#7872]) - runtime: wake deferred tasks before entering `block_in_place` ([#7879]) - sync: drop rx waker when oneshot receiver is dropped ([#7886]) - runtime: fix double increment of `num_idle_threads` on shutdown ([#7910], [#7918], [#7922]) Unstable -------- - fs: check for io-uring opcode support ([#7815]) - runtime: avoid lock acquisition after uring init ([#7850]) Documented ---------- - docs: update outdated unstable features section ([#7839]) - io: clarify the behavior of `AsyncWriteExt::shutdown()` ([#7908]) - io: explain how to flush stdout/stderr ([#7904]) - io: fix incorrect and confusing `AsyncWrite` documentation ([#7875]) - rt: clarify the documentation of `Runtime::spawn` ([#7803]) - rt: fix missing quotation in docs ([#7925]) - runtime: correct the default thread name in docs ([#7896]) - runtime: fix `event_interval` doc ([#7932]) - sync: clarify RwLock fairness documentation ([#7919]) - sync: clarify that `recv` returns `None` once closed and no more messages ([#7920]) - task: clarify when to use `spawn_blocking` vs dedicated threads ([#7923]) - task: doc that task drops before `JoinHandle` completion ([#7825]) - signal: guarantee that listeners never return `None` ([#7869]) - task: fix task module feature flags in docs ([#7891]) - task: fix two typos ([#7913]) - task: improve the docs of `Builder::spawn_local` ([#7828]) - time: add docs about auto-advance and when to use sleep ([#7858]) - util: fix typo in docs ([#7926]) [#7771]: tokio-rs/tokio#7771 [#7773]: tokio-rs/tokio#7773 [#7792]: tokio-rs/tokio#7792 [#7803]: tokio-rs/tokio#7803 [#7815]: tokio-rs/tokio#7815 [#7825]: tokio-rs/tokio#7825 [#7828]: tokio-rs/tokio#7828 [#7833]: tokio-rs/tokio#7833 [#7834]: tokio-rs/tokio#7834 [#7835]: tokio-rs/tokio#7835 [#7837]: tokio-rs/tokio#7837 [#7838]: tokio-rs/tokio#7838 [#7839]: tokio-rs/tokio#7839 [#7850]: tokio-rs/tokio#7850 [#7858]: tokio-rs/tokio#7858 [#7869]: tokio-rs/tokio#7869 [#7871]: tokio-rs/tokio#7871 [#7872]: tokio-rs/tokio#7872 [#7875]: tokio-rs/tokio#7875 [#7879]: tokio-rs/tokio#7879 [#7880]: tokio-rs/tokio#7880 [#7885]: tokio-rs/tokio#7885 [#7886]: tokio-rs/tokio#7886 [#7889]: tokio-rs/tokio#7889 [#7891]: tokio-rs/tokio#7891 [#7896]: tokio-rs/tokio#7896 [#7904]: tokio-rs/tokio#7904 [#7908]: tokio-rs/tokio#7908 [#7910]: tokio-rs/tokio#7910 [#7913]: tokio-rs/tokio#7913 [#7918]: tokio-rs/tokio#7918 [#7919]: tokio-rs/tokio#7919 [#7920]: tokio-rs/tokio#7920 [#7922]: tokio-rs/tokio#7922 [#7923]: tokio-rs/tokio#7923 [#7925]: tokio-rs/tokio#7925 [#7926]: tokio-rs/tokio#7926 [#7929]: tokio-rs/tokio#7929 [#7932]: tokio-rs/tokio#7932 Release notes: - 1.53.1: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.53.1 - 1.53.0: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.53.0 - 1.52.4: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.4 - 1.52.3: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.3 - 1.52.2: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.2 - 1.52.1: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.1 - 1.52.0: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.52.0 - 1.51.4: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.51.4 - 1.51.3: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.51.3 - 1.51.2: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.51.2 - 1.51.1: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.51.1 - 1.51.0: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.51.0 - 1.50.0: https://github.com/tokio-rs/tokio/releases/tag/tokio-1.50.0 --- Transitive Dependencies ----------------------- - libc (0.2.180 -> 0.2.189) - mio (1.1.1 -> 1.2.3) - socket2 (0.6.2 -> 0.6.5) - syn 3.0.6 (added) - tokio-macros (2.6.0 -> 2.7.2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Motivation
Worker threads in the multi-threaded runtime include a per-worker LIFO
slot which stores the last task notified by another task running on that
worker. This allows the last-notified task to be polled first when the
currently running task completes, decreasing latency in message-passing
"ping-pong" scenarios.
However --- as described in #4941 --- there's an issue with this that
can cause severe problems in some situations: the task in the LIFO slot
cannot be stolen by a work-stealing worker thread. This means that if a
task notifies another task and then goes CPU bound for a long period of
time without yielding, the notified task will never be able to execute
until the task that notified it can yield. This can result in a very
severe latency bubble in some scenarios. See, for instance, #4323,
#6954, oxidecomputer/omicron#8334, etc.
As a workaround, PR #4936 added an unstable
runtime::Builderoptionto disable the LIFO slot. However, this is a less-than-ideal
solution, as it means that applications which disable the LIFO slot due
to occasional usage patterns that cause latency bubbles when it is
enabled cannot benefit from the potential latency improvements it
offers in other usage patterns. And, it's an unstable option which the
user has to discover. In most cases, people whose programs contain usage
patterns that are pathological with regards to the LIFO slot don't know
this ahead of time: the typical narrative is that you write code that
happens to follow such a pattern, discover an unexpected latency spike
or hang in production, and then learn how to disable the LIFO slot. It
would be much nicer if the task in the LIFO slot could participate in
work-stealing like every other task.
Solution
This branch makes tasks in the LIFO slot stealable.
Broadly, I've taken the following approach:
Add a new
AtomicNotifiedtype that implements an atomicallyswappable cell storing a
Notifiedtask(1220253), and use it to represent
the LIFO slot instead of an
Option<Notified<Arc<Handle>>>. Thisway, other workers can take a task out of the LIFO slot while
work-stealing.
Move the LIFO slot out of the
worker::Corestruct and into the runqueue's
Innertype (75d8116),making it shared state between the
Localside of the queue owned bythe worker itself and the
Stealside used by remote workers tosteal from the queue.
There's a bunch of additional code in
worker::Corefor managingwhether to atually run a task from the LIFO slot or not. I opted
not to move any of this code into the run queue itself, as it
depends on other bits of internal worker state. Instead, we just
expose to the worker separate APIs for pushing/popping to/from the
main queue and for pushing/popping to/from the LIFO slot, resulting
in a fairly small diff to the worker's run loop.
Change the work-stealing code to also steal the LIFO task
(730a581 and
cb27dda). This actually turned out
to be pretty straightforwards: once we've stolen a chunk of tasks
from the targeted worker's run queue, we now also grab whatever's in
its' LIFO slot as well. If we stole a LIFO task, it's returned from
the
steal_intomethod in lieu of the first task in the run queue, sothat it gets to execute first, maintaining the latency improvement for
recently-notified tasks. This also was simpler than trying to wedge it
into the chunk of tasks to be pushed to the stealer's queue.
I've made the following test changes while working on this branch:
the LIFO slot is blocked from running when a task notifies it and then
blocks indefinitely. I've confirmed that this test fails on
masterand passes on this branch.
These are in addition to the existing work-stealing Loom tests, as
tasks notified by I/O or timers are still woken to the normal run
queue.
rt_unstable_metrics::worker_local_queue_depthintegration test,which was necessary as tasks in the LIFO slot now "count" towards the
worker's queue depth. We now have to make sure the no-op task that's
spawned has completed before spawning the tasks we actually attempt to
count, as it seems to sometimes end up in the LIFO slot and sometimes
not, causing the test to flake out.
Fixes #4941