Skip to content

Add missing ops up to Linux 6.15 #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Add missing ops up to Linux 6.15 #23

wants to merge 11 commits into from

Conversation

silvanshade
Copy link
Collaborator

@silvanshade silvanshade commented Jul 2, 2025 •
edited
Loading

This PR adds missing ops for io_uring up to Linux 6.15.

Related:

Opening as a Draft since there's still some remaining changes to make.

@silvanshade silvanshade force-pushed the linux-6.15 branch 8 times, most recently from a96a854 to fb85b91 Compare July 5, 2025 01:43
@silvanshade
Copy link
Collaborator Author

silvanshade commented Jul 5, 2025 •
edited
Loading

@sunfishcode I think this is ready for review. This has dependencies on the upstream PRs linked in the OP.

In addition to defining the missing ops, I've included at least one test for each of them, with the exception of RecvZc.

The RecvZc io_uring op currently only works on a few drivers and chipsets, notably:

Kernel Recipes 2024 - Efficient zero-copy networking using io_uring

  • bnxt (Thor chipset NICs, e.g., BCM957508)
  • gve
  • mlx5 (ConnectX-7 or newer, possibly ConnectX-6 Dx or Bluefield-2 with crypto)

EDIT: I was able to implement a test for ZCRX and verify that it works on an NVIDIA ConnectX-7.

I'm reluctant to add a test without being able to confirm it at least works on one system. We probably also should gate it behind some environment variables like the kernel selftests since it requires specific system configuration to even run.

I tried setting up the zcrx context on a ConnectX-5 but that didn't work because of lack of tcp-data-split and lack of rx-gro-hw. I will be able to try again with a Bluefield-2 with crypto offload probably next week (ConnectX-6 Dx equivalent), and if that doesn't work I'll try to get access to the BCM957508.

But if anyone else watching currently has compatible hardware, we could easily add a short test for it and verify it works.

In any case, it's a complex feature and may require more changes upstream, but probably shouldn't be a blocker for the rest of this PR and the others linked above.

@silvanshade silvanshade marked this pull request as ready for review July 5, 2025 02:26
@silvanshade silvanshade mentioned this pull request Jul 5, 2025
Open
@silvanshade silvanshade mentioned this pull request Jul 18, 2025
Open
@sunfishcode
Copy link
Collaborator

I unfortunately don't have the bandwidth to review this right now.

@silvanshade Would you be interested in helping maintain rustix-uring?

@jordanisaacs What would you think about adding a new maintainer here?

@silvanshade
Copy link
Collaborator Author

@sunfishcode Sure. I would be willing to help as a maintainer.

@jordanisaacs
Copy link
Owner

Hey @silvanshade. Thanks for helping! I invited you to the repo & crates.io so you can publish new releases.

jordanisaacs
jordanisaacs requested changes Jul 31, 2025
View reviewed changes
Copy link
Owner

@jordanisaacs jordanisaacs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! Looks good thanks for the tests. My main comment is around WaitId and the crates (patched rustix/a the linux sys).

src/opcode.rs
pub const CODE = sys::IoringOp::Waitid;

pub fn build(self) -> Entry {
use ::linux_raw_sys::general::{P_ALL, P_PGID, P_PID, P_PIDFD};
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the only use of linux_raw_sys? We shouldn't introduce a new crate, instead first upstream the missing components to rustix and then use those types.

src/types.rs
@@ -676,6 +679,29 @@ impl FutexWaitV {
}
}

#[non_exhaustive]
pub enum WaitId {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to earlier, not sure why we can't use the rustix WaitId.

Cargo.toml
@@ -19,11 +19,16 @@ members = [ "io-uring-test", "io-uring-bench" ]

[dependencies]
bitflags = { version = "2.4.0", default-features = false }
rustix = { version = "1.0.2", default-features = false, features = ["io_uring", "mm", "thread"] }
linux-raw-sys = { version = "0.10.0", default-features = false }
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets try and avoid adding any dependencies. This should really just be rustix + any supporting helpers.

Cargo.toml

[patch.crates-io]
linux-raw-sys = { git = "https://github.com/silvanshade/linux-raw-sys", branch = "io_uring-zcrx" }
rustix = { git = "https://github.com/silvanshade/rustix", branch = "linux-6.15-io_uring" }
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also lets only land this when there is a release we can point to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jordanisaacs jordanisaacs jordanisaacs requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@silvanshade @sunfishcode @jordanisaacs