Skip to content

Make control plane split by role work. - #18597

Open
cheftako wants to merge 9 commits into
kubernetes:masterfrom
cheftako:multipleControlRoles
Open

Make control plane split by role work.#18597
cheftako wants to merge 9 commits into
kubernetes:masterfrom
cheftako:multipleControlRoles

Conversation

@cheftako

@cheftako cheftako commented Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Create Etcd, Scheduler and Kube-Controller-Manager only IGs.
Able to create a cluser where the control plane is the composition of 4 IGs.
Enables separate APIServer, Etcd, Scheduler and KCM IGs.
CCM and Kops-Controller are then run on the APIServer.

@kubernetes-prow kubernetes-prow Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jul 15, 2026
@kubernetes-prow

kubernetes-prow Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign rifelpet for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow
kubernetes-prow Bot requested review from hakman and zetaab July 15, 2026 23:22
@kubernetes-prow kubernetes-prow Bot added area/nodeup area/provider/aws Issues or PRs related to aws provider area/provider/gcp Issues or PRs related to gcp provider area/provider/openstack Issues or PRs related to openstack provider labels Jul 15, 2026
@cheftako

cheftako commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

/test pull-kops-scenario-splitkcp-gcp

@cheftako
cheftako force-pushed the multipleControlRoles branch from 002f219 to 5872931 Compare July 15, 2026 23:29
@kubernetes-prow kubernetes-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 15, 2026
@cheftako
cheftako force-pushed the multipleControlRoles branch 3 times, most recently from d89d0fd to 16f1178 Compare July 16, 2026 05:04
@cheftako

cheftako commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

/test pull-kops-scenario-splitkcp-gcp

@cheftako

cheftako commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

/test pull-kops-scenario-splitkcp-gcp

1 similar comment
@cheftako

cheftako commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

/test pull-kops-scenario-splitkcp-gcp

@cheftako
cheftako force-pushed the multipleControlRoles branch from 82b9270 to e57277f Compare July 22, 2026 16:30
@cheftako

cheftako commented Jul 22, 2026

Copy link
Copy Markdown
Member Author

/test pull-kops-scenario-splitkcp-gcp

@kubernetes-prow kubernetes-prow Bot added area/channels needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jul 24, 2026
@cheftako
cheftako force-pushed the multipleControlRoles branch from bd97a83 to a99a585 Compare July 28, 2026 18:15
@kubernetes-prow kubernetes-prow Bot added area/documentation and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jul 28, 2026
@cheftako
cheftako force-pushed the multipleControlRoles branch 3 times, most recently from 1945c9b to 6b8ad59 Compare July 29, 2026 18:18
@cheftako

cheftako commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

/test pull-kops-e2e-k8s-aws-amazonvpc

@cheftako

cheftako commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

/test pull-kops-e2e-k8s-aws-calico

justinsb
justinsb reviewed Jul 29, 2026
View reviewed changes
Comment thread channels/pkg/cmd/apply_channel.go Outdated
Yes bool
Interval time.Duration
NodeName string
NodeLabels string

@justinsb justinsb Jul 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion:

  1. I like to add comments (or at least a comment) when adding a field that causes re-indentation of a struct. Because then the next update won't cause a full re-flow.

  2. I think this should be map[string]string . It's a pain to have that as a flag, so let's parse it as soon as possible (e.g. in RunE function of the cobra.Command)

justinsb
justinsb reviewed Jul 29, 2026
View reviewed changes
Comment thread cmd/kops-controller/controllers/node_controller.go
// If it is one of our managed labels, "prune" values we don't want to be there
switch k {
case nodelabels.RoleLabelAPIServer16, nodelabels.RoleLabelNode16, nodelabels.RoleLabelControlPlane20:
case nodelabels.RoleLabelAPIServer16,

@justinsb justinsb Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we define nodelabels.All or similar? I'm worried about "action at a distance" - too easy to forget to update this list when adding a new RoleLabel

@cheftako cheftako mentioned this pull request Jul 30, 2026
Merged
@kubernetes-prow kubernetes-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 30, 2026
cheftako added 6 commits August 2, 2026 23:22
@cheftako
[WIP] Initial work to make split control plane work.
c094210 
Got the IG creation working.
Kubelet is coming up on the etcd IG.
Currently having issues with getting the etcd to attach to peers.
fixed goimports
Fixed ptr to to new on count.
@cheftako
Minor tweaks to fix minor issues.
6a90c7f
@cheftako
Got a working cluster with split control plane.
12e32a5
@cheftako
Switched new kops role to be about workload not destination.
80479e7 
Fixed image rewrite to work for all of control plane.
Fixed docs to reflect changes.
Fixed role direct equality check.
Fix node check for bastion.
Fix KubControllerManager to KubeControllerManager.
@cheftako
Ran hack/update-expected.sh
3c5ea74 
Fixed experimental roles test.
Fixed nodelabel tests.
@cheftako
Fixing for changed function names.
ce27396
@cheftako
cheftako force-pushed the multipleControlRoles branch from 6b8ad59 to aeb19b6 Compare August 3, 2026 05:40
@kubernetes-prow kubernetes-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 3, 2026
@kubernetes-prow

kubernetes-prow Bot commented Aug 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

@cheftako: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kops-scenario-splitkcp-gcp e57277f link false /test pull-kops-scenario-splitkcp-gcp

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@cheftako
cheftako force-pushed the multipleControlRoles branch from aeb19b6 to dd340e2 Compare August 3, 2026 23:29
@cheftako cheftako changed the title [WIP] Initial work to make split control plane work. Make control plane split by role work. Aug 4, 2026
@kubernetes-prow kubernetes-prow Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@justinsb justinsb justinsb left review comments
@hakman hakman Awaiting requested review from hakman
@zetaab zetaab Awaiting requested review from zetaab

Assignees

No one assigned

Labels

area/addons area/api area/channels area/documentation area/kops-controller area/nodeup area/provider/aws Issues or PRs related to aws provider area/provider/gcp Issues or PRs related to gcp provider area/provider/openstack Issues or PRs related to openstack provider cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@cheftako @justinsb