fix: Keep sync_container_lifecycles() bgtask alive in a loop.
#2178
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Your org has enabled the Graphite merge queue for merging into mainAdd the label “flow:merge-queue” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “flow:hotfix” to add to the merge queue as a hot fix. You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link. |
Member
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This was referenced May 26, 2024
7 tasks
fregataa
changed the title
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
3 times, most recently
from
c1cd4fa to
20a9b6c
Compare
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
2 times, most recently
from
7a202ce to
a172e1a
Compare
7 tasks
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
3 times, most recently
from
df467c8 to
f784d6c
Compare
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
2 times, most recently
from
331b744 to
2ac6c4e
Compare
fregataa
changed the title
1 task
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
from
2ac6c4e to
7ac6b0e
Compare
This was referenced Jun 20, 2024
fregataa
changed the title
sync_container_lifecycles() bgtask alive in a loop.
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
2 times, most recently
from
bd7d1e4 to
25e70f3
Compare
3 tasks
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
4 times, most recently
from
cefd447 to
552b746
Compare
fregataa
force-pushed
the
topic/05-23-fix_enhanced_kernel_termination_handling
branch
from
552b746 to
59cbbc9
Compare
lablup-octodog
pushed a commit
that referenced
this pull request
Jul 5, 2024
Backported-from: main (24.09) Backported-to: 24.03 Backport-of: 2178
lablup-octodog
added a commit
that referenced
this pull request
Jul 5, 2024
… (#2394) Co-authored-by: Sanghun Lee <[email protected]> Backported-from: main (24.09) Backported-to: 24.03 Backport-of: 2178
1 task
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why
resource sync APIdepends on this PR?TL; DR
sync_containers_lifecycle()should workDetail
We have 3 different sources of Kernel / Container data
kernel_registryI tried to synchronize ALL Agent's data to Manager's DB but synchronizing actual containers to
kernel_registryor manager's data needs to process container destruction tasks which take a long time.Since
sync_containers_lifecycle()task synchronizes actual containers tokernel_registryperiodically in background, resource-sync API only needs to synckernel_registryto Manager side DB if it is ensured that the background task stays alive and works as we expect.Changes
Fix
sync_container_lifecycles()bgtask alive in a loop by handling any exceptions raised in the task.Enhance
sync_container_lifecycles()Checklist: (if applicable)