Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🚀 Pre-release master -> Staging_ResistanceIsFutile16 #888

Closed
17 tasks done
pcrespov opened this issue Feb 28, 2023 · 3 comments
Closed
17 tasks done

🚀 Pre-release master -> Staging_ResistanceIsFutile16 #888

pcrespov opened this issue Feb 28, 2023 · 3 comments
Assignees
@Surfict @pcrespov @sanderegg
Labels
release Preparation for pre-release/release
Milestone
Mithril

Comments

@pcrespov
Copy link
Member

pcrespov commented Feb 28, 2023 •
edited
Loading

In preparation for pre-release. Here an initial (incomplete) list of tasks to prepare before pre-releasing:

Motivation:

  • Weekly release
  • Ops maintenance: @Surfict

Devops check (⚠️ devops)

  • None in issues
  • @Surfict ops
    • mainly an SSD change
  • @sanderegg ops
    • change AMI ID for Autoscaling EC2_INSTANCES_AMI_ID
    • add custom scripts for Autoscaling EC2_INSTANCES_CUSTOM_BOOT_SCRIPTS
    "mkfs --type xfs -f -n ftype=1 /dev/nvme1n1","mkdir --parents /mnt/docker","mount /dev/nvme1n1 /mnt/docker","jq '.\"data-root\"=\"/mnt/docker/data\", .\"exec-root\"=\"/mnt/docker/exec\"' /etc/docker/daemon.json  > /tmp/daemon.json","mv /tmp/daemon.json /etc/docker/daemon.json","systemctl restart docker"
    • add pre-pulling entries for Autoscaling EC2_INSTANCES_PRE_PULL_IMAGES
    "itisfoundation/dynamic-sidecar:staging-github-latest", "itisfoundation/agent:staging-github-latest", "prom/node-exporter:v0.18.1", "portainer/agent:2.13.1", "itisfoundation/logspout:latest", "docker:20.10.14-dind", "caddy:2.4.5-alpine", "gcr.io/cadvisor/cadvisor:v0.45.0"
    • set REGISTRY_USER, REGISTRY_PW, REGISTRY_URL, REGISTRY_SSL and REGISTRY_AUTH for Autoscaling
    • NOTE: all these changes can be done after release was done.

e2e testing check

  • Mon. OK
  • Tue.

Commits (in order)

image

Full Changelog: ITISFoundation/osparc-simcore@staging_ResistanceIsFutile15...staging_ResistanceIsFutile16

Summary

{"start": "2023-02-28T12:30:00.000Z","end": "2023-02-28T16:30:00.000Z", "reason":"hardware maintenance + ResistanceIsFutile 16"}

Release

  • Release (release draft)
  • Check Release CI
  • Check deployed dalco/aws ()
  • Delete announcement
  • Check e2e runs
  • Announce
https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_ResistanceIsFutile16
@pcrespov pcrespov added the release Preparation for pre-release/release label Feb 28, 2023
@pcrespov pcrespov added this to the Mithril milestone Feb 28, 2023
@pcrespov
Copy link
Member Author

  • aws-staging did not deploy because auto-deployer-agent was off (replicas=0). @pcrespov reactivated (approved by @sanderegg ) and it deployed
  • aws-staging autoscale could not start. Set to zero and then back
  • @sanderegg still debugging aws-staging
  • @sanderegg deactivated e2e in aws-staging

@sanderegg
Copy link
Member

sanderegg commented Feb 28, 2023 •
edited
Loading

@mrnicegyu11 , @Surfict
some notes for devops that I encountered while setting Autoscaling ENVs in AWS-staging:

  • current active branch in https://github.com/ITISFoundation/osparc-ops-environments is staging_ResistanceIsFutile9, that is not very intuitive since there is also a branch staging_ResistanceIsFutile14?? Maybe we should review this stuff. it's not the end of the world but is a time waster.
  • REGISTRY_SSL is hard-coded to true in https://github.com/ITISFoundation/osparc-ops-environments instead of using the variables coming from https://git.speag.com/oSparc/osparc-ops-deployment-configuration which is set to false in staging.io branch (which is wrong by the way)
    image
  • NOTE for quotation marks escaping, after asking chatgpt, this guy actually helped me and this might interest you 
    my_string='This string contains both "double quotes" and '\''single quotes'\'''
echo $my_string
    I used that to set AUTOSCALING_EC2_INSTANCES_CUSTOM_BOOT_SCRIPTS
  • last but not least, since osparc-ops-environment in gitlab is currently archived I could not create issues. let's ask @pcrespov to unarchive it. Also I would propose to change its name (osparc-ops-issues?) and remove the files in there.

@sanderegg
Copy link
Member

e2e in aws-staging is activated again
autoscaling runs in aws-staging with pre-pulling of s4l-lite-core:2.0.106 at the moment, buffer set to 0

@pcrespov pcrespov closed this as completed Mar 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Surfict Surfict

@pcrespov pcrespov

@sanderegg sanderegg

Labels
release Preparation for pre-release/release
Projects
None yet
Milestone
Mithril
Development

No branches or pull requests
3 participants
@Surfict @pcrespov @sanderegg