Skip to content

Conversation

@fmassa
Copy link
Contributor

@fmassa fmassa commented Sep 1, 2025

In the previous implementation of get_log, finding the src placement corresponding to a given redistribution cost was annoying, as it required looking at the placement for the input node of a given operation.
This lead to sometimes hard to understand costs, as situations like the following happened quite often

x1 = op(x0)  # S(1) -> S(1), cost=[0]
x2 = alias(x1)   # S(0) -> S(0), cost=[10]

which could give the impression that there was a redistribution happening for the alias operator as S(0) -> S(0), but in fact it's actually its input which is being redistributed from S(1) to S(0).

This PR changes the representation to add additional comment lines, which explicitly mention the src and tgt placement for every redistribution.

The new implementation looks as follows:

# Redistributing for node permute_7:
# dtype_cast_5 = redistribute(dtype_cast_5, src=S(0)S(0), dst=RR)  # shape=(4096, 4096), comm_cost=525.239242769773

One drawback of the current implementation is that now we might have more lines in the file for different sharding decisions, which might make it harder to compare two sharding decisions. I might change this behavior prior to merging this PR.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 1, 2025
Copy link
Contributor

@ezyang ezyang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see debug print improvement, I stampy lol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants