🚀 Pre-release master -> staging_TheNameless2

[matusdrobuliak66]

What kind of pre-release?

master branch

Sprint Name

TheNameless

Pre-release version

2

Commit SHA

1d1c5d6

Did the commit CI suceeded?

• The commit CI succeeded.

Motivation

• Weekly release

What Changed

• ♻️ Feedback on payments plugin and service #4763 by @pcrespov
• Bump docker/setup-buildx-action from 2 to 3 #4769 by @dependabot
• ✨ dynamic-sidecar adding support for metrics and on_shutdown events for user services (⚠️OPS) #4697 by @GitHK
• ♻️On-demand clusters: remove dask-gateway #4760 by @sanderegg
• ✨ Product prices API and table (🗃️ ) #4758 by @pcrespov
• ✨🗃️ Is922/resource usage publish updated wallet credits #4762 by @matusdrobuliak66
• 🐛 Fixes link to payment form #4778 by @pcrespov
• 🐛Comp. backend: missing timestamps in log file #4771 by @sanderegg
• ♻️✨ Comp backend task state reporting fixed #4775 by @sanderegg
• 🐛Ensure the progress is set to None #4783 by @sanderegg

Devops check ⚠️ devops

• Graylog must be reconfigured (cd ./services/graylog && make configure) since dashboards were moved to content pack
• A new env DYNAMIC_SIDECAR_PROMETHEUS_SERVICE_LABELS was added but will be only used from next release

e2e testing check 🧪

No response

Summary 📝

• make release-staging name=TheNameless version=2 git_sha=1d1c5d6b727b9fc4d7cd0c246e990c900498922c
  • https://github.com/ITISFoundation/osparc-simcore/releases/new?prerelease=1&target=<commit_sha>&tag=staging_<sprint_name><version>&title=Staging%20<sprint_name><version>
• Draft pre-release
• Announce (add redis key maintenance in every concerned deployment)

  {"start": "2023-02-01T12:30:00.000Z", "end": "2023-02-01T13:00:00.000Z", "reason": "Release ResistanceIsFutile9 "}

• Announce release in Mattermost

  :loud_sound:  Maintenance scheduled for **NAMED_DAY DD. MM from START_TIME - END_TIME**.
  =========================================================================
  
  @all Be aware that you will automatically be logged out and your projects stopped and saved during the maintenance time. Affected:
  *   [https://staging.osparc.io](https://staging.osparc.io/)
  *   [https://https://staging.s4l-lite.io/](https://https://staging.s4l-lite.io//)
  
  and on premises:
  *   [https://osparc-staging.speag.com](https://osparc-staging.speag.com/)
  *   [https://tip-staging.speag.com](https://tip-staging.speag.com/)
  *   [https://s4l-staging.speag.com](https://s4l-staging.speag.com/)
  *   [https://s4l-lite-staging.speag.com](https://s4l-lite-staging.speag.com/)
  
  
  Reason: Scheduled staging-release of STAGING_NAME_AND_VERSION.
  
  Thanks for your understanding and sorry for the inconveniences,
  
  Your friendly oSparc Team
  
  

Releasing

• Check hanging sidecars. Helper command to run in director-v2 CLI simcore-service-director-v2 close-and-save-service <uuid>
• Release (release draft)
• Check Release CI
• Check deployed
  • aws deploy
  • dalco deploy
• Delete announcement
• Check e2e runs
• Announce

https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_<sprint_name><version>

[matusdrobuliak66]

These 2 problems occurred during release (it is already 3rd time we encountered them):

• The autoscaling service not started properly first time (we needed to restart it and than it worked correctly)

mportlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/scu/.venv/lib/python3.10/site-packages/simcore_service_autoscaling/main.py", line 11, in <module>
    the_settings = ApplicationSettings.create_from_envs()
  File "/home/scu/.venv/lib/python3.10/site-packages/settings_library/base.py", line 116, in create_from_envs
    return cls(**overrides)
  File "pydantic/env_settings.py", line 39, in pydantic.env_settings.BaseSettings.__init__
  File "pydantic/main.py", line 340, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1066, in pydantic.main.validate_model
  File "pydantic/fields.py", line 439, in pydantic.fields.ModelField.get_default
  File "/home/scu/.venv/lib/python3.10/site-packages/settings_library/base.py", line 30, in _default_factory
    return field_settings_cls()
  File "pydantic/env_settings.py", line 40, in pydantic.env_settings.BaseSettings.__init__
  File "pydantic/env_settings.py", line 75, in pydantic.env_settings.BaseSettings._build_values
  File "pydantic/env_settings.py", line 200, in pydantic.env_settings.EnvSettingsSource.__call__
pydantic.env_settings.SettingsError: error parsing env var "NODES_MONITORING_NEW_NODES_LABELS"

• The webserver service not started properly first time (we needed to restart it and than it worked correctly)

INFO: [boot.sh]  Selected config server-docker-prod.yaml
INFO: [boot.sh]  Log-level app/server: INFO/info
[2023-09-22 08:30:23 +0000] [7] [INFO] Starting gunicorn 20.1.0
[2023-09-22 08:30:23 +0000] [7] [INFO] Listening at: http://0.0.0.0:8080 (7)
[2023-09-22 08:30:23 +0000] [7] [INFO] Using worker: aiohttp.GunicornWebWorker
[2023-09-22 08:30:23 +0000] [763] [INFO] Booting worker with pid: 763
WEBSERVER_META_MODELING still under development and will be disabled.
WEBSERVER_VERSION_CONTROL still under development and will be disabled.
[2023-09-22 08:31:09 +0000] [7] [INFO] Handling signal: term

and inspect

Health
FailingStreak 6
Log
0 { End: 2023-09-22T08:30:22.356530741Z, ExitCode: -1, Output: Health check exceeded timeout (5s), Start: 2023-09-22T08:30:17.355617169Z }
1 { End: 2023-09-22T08:30:37.383842061Z, ExitCode: -1, Output: Health check exceeded timeout (5s), Start: 2023-09-22T08:30:32.371954541Z }
2 { End: 2023-09-22T08:30:52.39780715Z, ExitCode: -1, Output: Health check exceeded timeout (5s), Start: 2023-09-22T08:30:47.397343233Z }
3 { End: 2023-09-22T08:31:07.414604601Z, ExitCode: -1, Output: Health check exceeded timeout (5s), Start: 2023-09-22T08:31:02.408183193Z }
4 { End: 2023-09-22T08:31:19.560340547Z, ExitCode: 137, Output: , Start: 2023-09-22T08:31:17.425380463Z }
Status unhealthy