Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: consolidate static and elastic scheduling strategy #253

Merged
merged 3 commits into from
Mar 1, 2024

Conversation

D-D-H
Copy link
Contributor

@D-D-H D-D-H commented Feb 5, 2024

Currently, Jifa supports two types of clusters:

  • static: all worker nodes are fixed and started up in advance. A static worker node has self-storage.
  • elastic: worker nodes are scheduled dynamically based on k8s. The storage is shared between master nodes and worker nodes.

This feature introduces a hybrid type to consolidate these two types so that a Jifa cluster can have static and elastic worker nodes at the same time. The location of a new file is decided by the rules (see FileLocationRuleEntity).

This feature doesn't eliminate the storage limitation(which could be a follow-up feature):

  • static worker nodes have an independent storage.
  • master and elastic worker nodes share shared storage.

@D-D-H D-D-H marked this pull request as draft February 5, 2024 14:37
@D-D-H D-D-H self-assigned this Feb 6, 2024
@D-D-H D-D-H force-pushed the hybrid_scheduling_strategy branch from d83f10f to acbd44c Compare February 14, 2024 15:21
@y1yang0
Copy link
Contributor

y1yang0 commented Feb 20, 2024

Looks good

@D-D-H D-D-H marked this pull request as ready for review February 24, 2024 00:32
@D-D-H D-D-H merged commit f00c273 into main Mar 1, 2024
4 checks passed
@D-D-H D-D-H deleted the hybrid_scheduling_strategy branch March 14, 2024 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants