Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: TCP socket support using LibUV #6683

Draft
wants to merge 29 commits into
base: master
Choose a base branch
from

Conversation

algebraic-dev
Copy link
Contributor

This PR introduces TCP socket support using the LibUV library, enabling asynchronous I/O operations with it.

algebraic-dev and others added 11 commits January 10, 2025 16:44
This PR adds support for `Timer` and a event loop thread that gets
requests from another threads and executes it.

---------

Co-authored-by: Markus Himmel <[email protected]>
Co-authored-by: Henrik Böving <[email protected]>
This PR implements a basic asynchronous timer API on top of the libuv
work.

It purposely puts this into `Std.Internal` as we might still have to
change the API as we continue develop of the async library across
releases so I would only like to stabilize it once we are certain this
is a fine API.

A few additional notes:
- we currently do not implement a bind operator on `AsyncTask` on
purpose as `Task.bind` on `Task.pure` is a non trivial operation and
users should be aware of it. Furthermore there is the consideration that
as they will have to bind on both `IO` and `AsyncTask` we might want to
make potential task points explicit in the syntax (did somebody say
`await`?).
- the API generally takes inspiration from
https://docs.rs/tokio/latest/tokio/time/index.html, though it has to
adapt as Rust's and Lean's asynchronity concepts are sufficiently
different.

Stacked on top of leanprover#6219.
@algebraic-dev algebraic-dev requested a review from TwoFX as a code owner January 17, 2025 23:53
@algebraic-dev algebraic-dev marked this pull request as draft January 17, 2025 23:53
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Jan 23, 2025
@leanprover-community-bot
Copy link
Collaborator

leanprover-community-bot commented Jan 23, 2025

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 9b629cc81f01642200eb0762da39b2935652754c --onto 9b74c07767dc50645efa00356a7724e7f7176227. (2025-01-23 05:57:38)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 104b3519d79f5996dd6d325cba18d94a4615e418 --onto 20c616503abe5ce4253c56dbcd7766a91c675ba0. (2025-01-30 00:33:23)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 104b3519d79f5996dd6d325cba18d94a4615e418 --onto a35bf7ee4c4d900475d88886825a5337f389bd0f. (2025-01-30 00:52:24)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 104b3519d79f5996dd6d325cba18d94a4615e418 --onto e922edfc218090ab2e54092336c67fe3b970dfc2. (2025-01-30 17:26:52)

Copy link
Contributor

@hargoniX hargoniX left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is very promising now! There's mostly on memory leaks and stylistic things left but the implementation looks like it should be able to work now.

Beyond these issues it would of course be great if we could also have some tests like for Timer that exercise this API. I would assume you can just connect to localhost and do an echo or something like that? Of course we should have tests for both IPv4 and v6.

After we have those tests I guess it might also be a good idea to write a little benchmark to see how many TCP connections we can handle per second to get an estimate of what kind of load we can expect to handle at best given this implementation.

src/Std/Internal/UV/Tcp.lean Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Outdated Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
src/runtime/uv/tcp.cpp Show resolved Hide resolved
}

/* Std.Internal.UV.Tcp.Socket.recv (socket : @& Socket) : IO (IO.Promise (Except IO.Error ByteArray)) */
extern "C" LEAN_EXPORT lean_obj_res lean_uv_tcp_recv(b_obj_arg socket) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we seem to be approaching a final version of this now. It would be great if we could pass a size : USize into recv to tell the size of the buf to allocate instead of always taking libuvs suggestion. That should be doable right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants