Skip to content

Add LLImpl pack/unpack over LLPacket geometry (#3554) - #3554

Closed
function47 wants to merge 1 commit into
meta-pytorch:mainfrom
function47:export-D110940544
Closed

Add LLImpl pack/unpack over LLPacket geometry (#3554)#3554
function47 wants to merge 1 commit into
meta-pytorch:mainfrom
function47:export-D110940544

Conversation

@function47

@function47 function47 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary:

Second llx building block: LLImpl<P>, the ThreadGroup-driven encode/decode of
a byte range into/out of an LLPacket-formatted staging region.

  • pack(group, staging, src, nbytes, flagVal): encode the payload into
    consecutive packets, stamping each packet's trailing flag with flagVal.
  • unpack(group, dst, staging, nbytes, flagVal): spin until each owned
    packet's flag == flagVal, then decode the payload out.
  • Flag I/O (store_flag / load_flag / is_flag_set): the flag sits entirely
    inside P::kFlagLane's 16 B slot, so it is read/written as ONE wide volatile
    transfer instead of a per-word scalar loop. flagVal is replicated across the
    full flag width, so a torn transfer that carries the new value in only part of
    the flag fails is_flag_setunpack never accepts half-arrived data.

Parallelism (v1): one thread owns one whole packet, grid-strided across the
group. pack writes each packet's trailing flag last; on the wire the whole
staging region is fenced and RDMA-put as one unit, so the flag is never observed
before its data. unpack spins only on the packets a thread owns (no
group-wide readiness barrier), then decodes.

Additive only: new header comms/prims/core/LLImpl.cuh behind a new
//comms/prims:ll_impl target, plus //comms/prims/tests:ll_impl_test and its
:ll_impl_test_kernels companion. No existing code path is modified.

Reviewed By: snarayankh

Differential Revision: D110940544

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Aug 10, 2026
@meta-codesync

meta-codesync Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

@function47 has exported this pull request. If you are a Meta employee, you can view the originating Diff in D110940544.

@meta-codesync meta-codesync Bot changed the title Add LLImpl pack/unpack over LLPacket geometry Add LLImpl pack/unpack over LLPacket geometry (#3554) Aug 10, 2026
function47 added a commit to function47/torchcomms-1 that referenced this pull request Aug 10, 2026
Summary:

Second llx building block: `LLImpl<P>`, the ThreadGroup-driven encode/decode of
a byte range into/out of an `LLPacket`-formatted staging region.

- `pack(group, staging, src, nbytes, flagVal)`: encode the payload into
  consecutive packets, stamping each packet's trailing flag with `flagVal`.
- `unpack(group, dst, staging, nbytes, flagVal)`: spin until each owned
  packet's flag == `flagVal`, then decode the payload out.
- Flag I/O (`store_flag` / `load_flag` / `is_flag_set`): the flag sits entirely
  inside `P::kFlagLane`'s 16 B slot, so it is read/written as ONE wide volatile
  transfer instead of a per-word scalar loop. `flagVal` is replicated across the
  full flag width, so a torn transfer that carries the new value in only part of
  the flag fails `is_flag_set` — `unpack` never accepts half-arrived data.

Parallelism (v1): one thread owns one whole packet, grid-strided across the
group. `pack` writes each packet's trailing flag last; on the wire the whole
staging region is fenced and RDMA-put as one unit, so the flag is never observed
before its data. `unpack` spins only on the packets a thread owns (no
group-wide readiness barrier), then decodes.

Additive only: new header `comms/prims/core/LLImpl.cuh` behind a new
`//comms/prims:ll_impl` target, plus `//comms/prims/tests:ll_impl_test` and its
`:ll_impl_test_kernels` companion. No existing code path is modified.

Reviewed By: snarayankh

Differential Revision: D110940544
function47 added a commit to function47/torchcomms-1 that referenced this pull request Aug 10, 2026
Summary:

Second llx building block: `LLImpl<P>`, the ThreadGroup-driven encode/decode of
a byte range into/out of an `LLPacket`-formatted staging region.

- `pack(group, staging, src, nbytes, flagVal)`: encode the payload into
  consecutive packets, stamping each packet's trailing flag with `flagVal`.
- `unpack(group, dst, staging, nbytes, flagVal)`: spin until each owned
  packet's flag == `flagVal`, then decode the payload out.
- Flag I/O (`store_flag` / `load_flag` / `is_flag_set`): the flag sits entirely
  inside `P::kFlagLane`'s 16 B slot, so it is read/written as ONE wide volatile
  transfer instead of a per-word scalar loop. `flagVal` is replicated across the
  full flag width, so a torn transfer that carries the new value in only part of
  the flag fails `is_flag_set` — `unpack` never accepts half-arrived data.

Parallelism (v1): one thread owns one whole packet, grid-strided across the
group. `pack` writes each packet's trailing flag last; on the wire the whole
staging region is fenced and RDMA-put as one unit, so the flag is never observed
before its data. `unpack` spins only on the packets a thread owns (no
group-wide readiness barrier), then decodes.

Additive only: new header `comms/prims/core/LLImpl.cuh` behind a new
`//comms/prims:ll_impl` target, plus `//comms/prims/tests:ll_impl_test` and its
`:ll_impl_test_kernels` companion. No existing code path is modified.

Reviewed By: snarayankh

Differential Revision: D110940544
Summary:

Second llx building block: `LLImpl<P>`, the ThreadGroup-driven encode/decode of
a byte range into/out of an `LLPacket`-formatted staging region.

- `pack(group, staging, src, nbytes, flagVal)`: encode the payload into
  consecutive packets, stamping each packet's trailing flag with `flagVal`.
- `unpack(group, dst, staging, nbytes, flagVal)`: spin until each owned
  packet's flag == `flagVal`, then decode the payload out.
- Flag I/O (`store_flag` / `load_flag` / `is_flag_set`): the flag sits entirely
  inside `P::kFlagLane`'s 16 B slot, so it is read/written as ONE wide volatile
  transfer instead of a per-word scalar loop. `flagVal` is replicated across the
  full flag width, so a torn transfer that carries the new value in only part of
  the flag fails `is_flag_set` — `unpack` never accepts half-arrived data.

Parallelism (v1): one thread owns one whole packet, grid-strided across the
group. `pack` writes each packet's trailing flag last; on the wire the whole
staging region is fenced and RDMA-put as one unit, so the flag is never observed
before its data. `unpack` spins only on the packets a thread owns (no
group-wide readiness barrier), then decodes.

Additive only: new header `comms/prims/core/LLImpl.cuh` behind a new
`//comms/prims:ll_impl` target, plus `//comms/prims/tests:ll_impl_test` and its
`:ll_impl_test_kernels` companion. No existing code path is modified.

Reviewed By: snarayankh

Differential Revision: D110940544
@meta-codesync

meta-codesync Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

This pull request has been merged in a66c6f6.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. Merged meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant