Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Autoscaling: connect with rabbitmq #3620

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Nov 29, 2022

What do these changes do?

This PR connects the autoscaling app with RabbitMQ.

  • added GET /status to recover the current connection status with rabbit MQ (will be enhanced later with EC2, maybe Redis)
  • tasks created through the dynamic sidecar also get the labels (user_id, study_id, uuid) NOTE: this will be at some point refactored to better names using inverse urls (see #3638).
  • added function to ping rabbitMQ connection
  • removed default values in settings RabbitSettings (this could cause some issues in the deployments - to be seen)
  • prepare for simplification of tests using rabbitMQ by having a fixture that creates a rabbitmq client
  • remove dependency to simcore-postgres-database package from rabbitmq_messages.py
  • light refactoring

Bonus:

Related issue/s

needed for #3341

How to test

Checklist

@sanderegg sanderegg self-assigned this Nov 29, 2022
@sanderegg sanderegg added this to the Athena milestone Nov 29, 2022
@codecov
Copy link

codecov bot commented Nov 29, 2022

Codecov Report

Merging #3620 (698a0f0) into master (57c8805) will increase coverage by 2.0%.
The diff coverage is 97.8%.

❗ Current head 698a0f0 differs from pull request most recent head f02d3c7. Consider uploading reports for the commit f02d3c7 to get more accurate results

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #3620     +/-   ##
========================================
+ Coverage    81.8%   83.9%   +2.0%     
========================================
  Files         868     871      +3     
  Lines       36555   36658    +103     
  Branches      782     782             
========================================
+ Hits        29922   30771    +849     
+ Misses       6426    5680    -746     
  Partials      207     207             
Flag Coverage Δ
integrationtests 65.6% <100.0%> (-11.3%) ⬇️
unittests 81.4% <97.8%> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...es/settings-library/src/settings_library/rabbit.py 0.0% <0.0%> (ø)
...rector_v2/modules/comp_scheduler/dask_scheduler.py 89.4% <ø> (+11.5%) ⬆️
...es/dynamic_sidecar/docker_service_specs/sidecar.py 87.5% <ø> (+8.3%) ⬆️
...s/models-library/src/models_library/basic_regex.py 100.0% <100.0%> (ø)
...ckages/models-library/src/models_library/docker.py 100.0% <100.0%> (ø)
...ls-library/src/models_library/rabbitmq_messages.py 97.8% <100.0%> (+0.6%) ⬆️
...ackages/service-library/src/servicelib/rabbitmq.py 94.5% <100.0%> (+0.3%) ⬆️
...ervice_autoscaling/api/dependencies/application.py 100.0% <100.0%> (ø)
...ling/src/simcore_service_autoscaling/api/health.py 100.0% <100.0%> (ø)
...ling/src/simcore_service_autoscaling/api/routes.py 100.0% <100.0%> (ø)
... and 61 more

@sanderegg sanderegg force-pushed the enhancement/autoscaling/connect_with_rabbitmq branch 2 times, most recently from 48dca77 to a44b36a Compare November 30, 2022 08:25
@sanderegg sanderegg force-pushed the enhancement/autoscaling/connect_with_rabbitmq branch 4 times, most recently from f4f1f63 to ff71907 Compare December 5, 2022 07:34
@sanderegg sanderegg force-pushed the enhancement/autoscaling/connect_with_rabbitmq branch from aaadd25 to 4377e7e Compare December 5, 2022 18:23
@sanderegg sanderegg marked this pull request as ready for review December 5, 2022 18:32
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice: one question below about rabbit being not properly configured. I'm wondering what happens if there was a mistake in the DNS, when and where would that be seen

@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 6, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sanderegg sanderegg merged commit 3701d49 into ITISFoundation:master Dec 6, 2022
@sanderegg sanderegg deleted the enhancement/autoscaling/connect_with_rabbitmq branch December 6, 2022 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky tests/integration/02/test_rabbit.py::test_publish_about_users_project
3 participants