Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ways the HUD compilers dashboard could be better #144890

Open
zou3519 opened this issue Jan 15, 2025 · 1 comment
Open

Ways the HUD compilers dashboard could be better #144890

zou3519 opened this issue Jan 15, 2025 · 1 comment
Labels
enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: devx Related to PyTorch contribution experience (HUD, pytorchbot) triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@zou3519
Copy link
Contributor

zou3519 commented Jan 15, 2025

I got here because I'm trying to answer the question of "which compiler benchmarks regressed in the past year?" I've spent a couple of hours on the HUD dashboard page, and I still haven't figured this out yet. Here's some of the gripes that I ran into while trying to answer this question.

  1. The page seems to refresh itself every couple of minutes. This disrupts the train of thought. Also, I am not sure if the settings change when it refreshes.
  2. The passrate chart and the graphs don't have all of the data. In particular, the passrate chart doesn't contain the max_autotune configs. I don't know how to actually click into the max_autotune data.

Image

  1. Wrong datapoints on PT2 dashboard for retried workflows test-infra#6173
  2. There's one passrate chart but there are 3 passrate graphs. Scrolling between the graphs is kind of annoying
  3. The graphs have so many series that some of them are hidden. Might be nicer to increase the height?

Image

  1. It's not clear to me how to hack on these charts. Using our internal tools (like scuba and unidash), it's easy (and well-known) on how to look up information.

Hypothesis: If we feed the data to internal sources and use internal tooling as the UXs, then we would be more productive than trying to roll our own UX.

cc @ZainRizvi @kit1980 @huydhn @clee2000

@huydhn
Copy link
Contributor

huydhn commented Jan 15, 2025

cc @yangw-dev

(2) is the same as pytorch/test-infra#6175

@malfet malfet added enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: devx Related to PyTorch contribution experience (HUD, pytorchbot) triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels Jan 16, 2025
@ZainRizvi ZainRizvi moved this to Cold Storage in PyTorch OSS Dev Infra Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Not as big of a feature, but technically not a bug. Should be easy to fix module: devx Related to PyTorch contribution experience (HUD, pytorchbot) triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
Status: Cold Storage
Development

No branches or pull requests

3 participants