Skip to content

Conversation

@azreenz
Copy link
Collaborator

@azreenz azreenz commented Jun 3, 2025

Add visualization tool for topology cli to create ASCII art to visualize block and tree slurm topologies

  • Add --generate_visualization option (may want to change how this is named)
  • Add --visual_block_size option to determine what each block is supposed to look like (default as 18 to mimic a rack)
  • Add visualize function to Topology class
  • run function now returns the topology string so visualize function can take it in to generate the visualization
  • If nodes are less than max_block_size then we fill out with 'X'
  • Commented out blocks are still visualized but include a "** This block is ineligible for scheduling because # of nodes < min block size**" line in the header
  • Add test cases to validate new function

Example Block topology visual:

block 1  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f723c
|-----------------|-----------------|-----------------|
| hpcbench-hpc-1  | hpcbench-hpc-5  | hpcbench-hpc-9  |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-13 | hpcbench-hpc-17 | hpcbench-hpc-21 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-25 | hpcbench-hpc-29 | hpcbench-hpc-33 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-37 | hpcbench-hpc-41 | hpcbench-hpc-45 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-49 | hpcbench-hpc-53 | hpcbench-hpc-57 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-61 | hpcbench-hpc-65 | hpcbench-hpc-69 |
|-----------------|-----------------|-----------------|

block 2  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f721e
|-----------------|-----------------|-----------------|
| hpcbench-hpc-2  | hpcbench-hpc-6  | hpcbench-hpc-10 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-14 | hpcbench-hpc-18 | hpcbench-hpc-22 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-26 | hpcbench-hpc-30 | hpcbench-hpc-34 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-38 | hpcbench-hpc-42 | hpcbench-hpc-46 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-50 | hpcbench-hpc-54 | hpcbench-hpc-58 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-62 | hpcbench-hpc-66 | hpcbench-hpc-70 |
|-----------------|-----------------|-----------------|

block 3  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f724b
|-----------------|-----------------|-----------------|
| hpcbench-hpc-3  | hpcbench-hpc-7  | hpcbench-hpc-11 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-15 | hpcbench-hpc-19 | hpcbench-hpc-23 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-27 | hpcbench-hpc-31 | hpcbench-hpc-35 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-39 | hpcbench-hpc-43 | hpcbench-hpc-47 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-51 | hpcbench-hpc-55 | hpcbench-hpc-59 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-63 | hpcbench-hpc-67 | hpcbench-hpc-71 |
|-----------------|-----------------|-----------------|

block 4  : # of Nodes = 18
ClusterUUID + CliqueID : 5e797fc6-0f46-421a-8724-0e102c0f722a
|-----------------|-----------------|-----------------|
| hpcbench-hpc-4  | hpcbench-hpc-8  | hpcbench-hpc-12 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-16 | hpcbench-hpc-20 | hpcbench-hpc-24 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-28 | hpcbench-hpc-32 | hpcbench-hpc-36 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-40 | hpcbench-hpc-44 | hpcbench-hpc-48 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-52 | hpcbench-hpc-56 | hpcbench-hpc-60 |
|-----------------|-----------------|-----------------|
| hpcbench-hpc-64 | hpcbench-hpc-68 | hpcbench-hpc-72 |
|-----------------|-----------------|-----------------|

Example Block topology visualization with nodes less than max_block_size

block 1  : # of Nodes = 6
ClusterUUID + CliqueID : N/A N/A
|---------------|---------------|---------------|
| vis0603-gpu-2 | vis0603-gpu-5 | vis0603-gpu-1 |
|---------------|---------------|---------------|
| vis0603-gpu-3 | vis0603-gpu-4 | vis0603-gpu-6 |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|

Example Block Topology visual with block with nodes less than min block size

block 1  : # of Nodes = 6
ClusterUUID + CliqueID : N/A N/A
** This block is ineligible for scheduling because # of nodes < min block size 8**
|---------------|---------------|---------------|
| vis0603-gpu-4 | vis0603-gpu-2 | vis0603-gpu-5 |
|---------------|---------------|---------------|
| vis0603-gpu-1 | vis0603-gpu-6 | vis0603-gpu-3 |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|
|       X       |       X       |       X       |
|---------------|---------------|---------------|

Example Tree topology visual:

Switch 5 (root)
├── Switch 0 (3 nodes)
│   ├── hpcbench-hpc-36
│   ├── hpcbench-hpc-39
│   └── hpcbench-hpc-42
├── Switch 1 (6 nodes)
│   ├── hpcbench-hpc-1
│   ├── hpcbench-hpc-35
│   ├── hpcbench-hpc-38
│   ├── hpcbench-hpc-41
│   ├── hpcbench-hpc-44
│   └── hpcbench-hpc-49
├── Switch 2 (3 nodes)
│   ├── hpcbench-hpc-37
│   ├── hpcbench-hpc-45
│   └── hpcbench-hpc-46
├── Switch 3 (2 nodes)
│   ├── hpcbench-hpc-40
│   └── hpcbench-hpc-43
└── Switch 4 (2 nodes)
    ├── hpcbench-hpc-47
    └── hpcbench-hpc-48

@azreenz azreenz requested review from aditigaur4 and bwatrous June 4, 2025 16:20
@azreenz azreenz marked this pull request as ready for review June 4, 2025 16:20
@azreenz azreenz force-pushed the azreenzaman/topology-visualization branch 2 times, most recently from 4a212bf to 3f08f03 Compare June 4, 2025 18:56
bwatrous
bwatrous previously approved these changes Jun 4, 2025
str: ASCII visualization of the topology.
"""

if self.topo_type == TopologyType.BLOCK:
Copy link
Collaborator

@aditigaur4 aditigaur4 Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you have extremely big if else blocks-- its always best to condense them into functions.

if self.topo_type == TopologyType.BLOCK:
    visualize_block_topology()
else: 
    # assuming tree
    visualize_tree_topology()

Copy link
Collaborator

@aditigaur4 aditigaur4 Jun 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also be nice if visualize_topology* functions actually get moved to util since its not really part of core topology logic. You also dont really need any data from the topology class. Just the string and the block size. Thats an optional suggestion though.

@azreenz azreenz force-pushed the azreenzaman/topology-visualization branch from 3f08f03 to 14d5ba0 Compare June 30, 2025 15:17
@azreenz azreenz force-pushed the azreenzaman/topology-visualization branch from 14d5ba0 to 8d018cb Compare June 30, 2025 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants