Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add shared memory transport and refactor metadata format #147

Closed
wants to merge 2 commits into from

Conversation

alogfans
Copy link
Collaborator

@alogfans alogfans commented Mar 17, 2025

This patch is the first stage of Transfer Engine's code refactoring.

Regarding that in many AI data centers, there are multiple link between two storage medias, we want Transfer Engine to use optimal link to reach higher throughput. E.g., two processes in one machine may use shared memory or nvlink transports.

But the current implementation requires to use EITHER rdma or TCP, which is not suitable for extension. Also, there are many dead code in the transport code. So I plan to refact the metadata & transport impl code to make Transfer Engine more organizable.

Overally, this patch changes the following things:

  • Metadata format, see mooncake-transfer-engine/include/segment.h, we also extract some code for coding & decoding segment desc in the seperte file.
  • We uniform terms SegmentID and SegmentHandle. In the future, we will remove the term local_server_name and replace it with local_segment_name to avoid ambiguous.
  • Add shared memory transports, currently only enable local-to-local data transfer, the remaining part will be fixed at stage 2
  • Remove dead code

@alogfans alogfans force-pushed the transport-refactor-stage-1 branch from 0b17005 to 18d08a6 Compare March 20, 2025 12:12
@alogfans alogfans changed the title [Feature] Add shared memory transport [Feature] Add shared memory transport and refactor metadata format Mar 20, 2025
@alogfans
Copy link
Collaborator Author

This patch is overrided by #170.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant