Skip to content

Enable autoscaling #128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 110 commits into from
Closed

Enable autoscaling #128

wants to merge 110 commits into from

Conversation

sjpb
Copy link
Collaborator

@sjpb sjpb commented Oct 15, 2021 •
edited
Loading

Enables autoscaling.

Requires these PRs to be merged first (probably in this order):

NB: requirements.yml here should be modified once that's happened

TODOs

  • Create cluster against JG's specs (i.e. no "normal" nodes.
  • Add testing via smslabs CI
  • Add slurm <-> cloud reconciliation
  • Add ports (and security groups?)
  • Refactor CI for speed?

Notes for release notes

  • Autoscaling added
  • Memory % changed?

sjpb added 30 commits September 8, 2021 12:14
@sjpb
WIP autoscale PoC
a123492
@sjpb
add IMB package to allow testing
c9c9bfc
@sjpb
move cloud_nodes config to right environment
ab14526
@sjpb
fix /etc/openstack permissions for resume
67b16a4
@sjpb
fix clouds.yaml
a618aca
@sjpb
get resume/suspend scripts working manually
341a5c9
@sjpb
note issue with adhoc slurm restart for combined headnode
99fe7ad
@sjpb
fix openhpc variables for autoscale
a956a54
@sjpb
set new image ID
4ea81c5
@sjpb
set autoscale branch for openhpc role requirements
354c67a
@sjpb
fix /etc/openstack for autoscale
c74a271
@sjpb
remove SlurmctldParameters unsupported in slurm 20.02.5
73eed39
@sjpb
use openhpc_munge_key parameter
967d107
@sjpb
don't cache node ips in slurm
94de099
@sjpb
tune slurm debug info for powersave only
99793ad
@sjpb
use default security groups
1a3fd48
@sjpb
remove ssh proxying from inventory
b992161
@sjpb
add helloworld MPI program setup
0ebba20
@sjpb
specify NFS server by hostname not IP
79b0516
@sjpb
update to latest built image
9f9430a
@sjpb
remove inventory hosts file from git
95a8ed2
@sjpb
show cloud nodes even when powered off
510a1bf
@sjpb
revert compute image to vanilla cento8.2
9392c39
@sjpb
remove sausagecloud environment
9467973
@sjpb
move autoscale into slurm
000a4e7
@sjpb
allow for overriding slurm config in appliance
f6514e6
@sjpb
add autoscale group/group_vars
b285620
@sjpb
use autoscale branch of openhpc role
7ae5042
@sjpb
Add podman_cidr to allow changing podman network range
ea5c3bc
@sjpb
Merge branch 'main' into feature/autoscale
9fba933
@sjpb sjpb marked this pull request as draft October 15, 2021 08:10
@sjpb sjpb requested a review from jovial December 10, 2021 08:24
sjpb added 23 commits January 24, 2022 15:37
@sjpb
Merge branch 'main' into feature/autoscale
4a342fd
@sjpb
Merge branch 'main' into feature/autoscale
2e9c926
@sjpb
fix up autoscale test environment
2c6c642
@sjpb
change autoscale group to be openstack-specific
12e7de4
@sjpb
fix security groups in smslabs for idempotency
c0370d6
@sjpb
fix smslabs env not being configless, add checks for this
2192ab7
@sjpb
WIP for smslabs autoscale
0290115
@sjpb
add basic autoscale to CI
7c33f1c
@sjpb
fix failure during CI at 'stackhpc.slurm_openstack_tools.autoscale : …
e9a0552 
…Modify openhpc_slurm_partitions'
@sjpb
fix cloud instance name in CI
2414203
@sjpb
fix cloud network definition during CI image build
e649ab8
@sjpb
remove debugging exit
c9e956e
@sjpb
smslabs CI fix for cloud node image
69e5d07
@sjpb
fix HPL-solo issue due to memory mismatch with 2 static + 2 cloud nod…
b41b4ae 
…es in smslabs CI
@sjpb
re-fix security groups in smslabs for idempotency
0e597f8
@sjpb
smslabs: don't require OS_CLOUD= to be set for TF
efc7ab4
@sjpb
update smslabs CI to do build in parallel with deploy
776db15
@sjpb
add squid proxy for smslabs
4b3ac96
@sjpb
move pytools to feature/ports
f6b3efa
@sjpb
use CI project squid for smslabs
2e3c77b
@sjpb
change smslabs CI workflow name for clarity
0f7dc83
@sjpb
disable dnf proxy in smslabs for debugging
c217f20
@sjpb
add check_slurm tasks to sms-labs CI
24bfeb2
@sjpb sjpb mentioned this pull request Mar 8, 2022
Open
@sjpb
Copy link
Collaborator Author

sjpb commented Mar 9, 2022

Superseded by #151

@sjpb sjpb closed this Mar 9, 2022
@sjpb sjpb deleted the feature/autoscale branch March 15, 2022 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jovial jovial Awaiting requested review from jovial

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@sjpb