Repository navigation
refactor!: migrate to the operator-go v0.13.0 Gen 3 framework - #268
Merged
Merged
Conversation
Replaces the Gen 2b architecture (BaseCluster + per-role packages, pinned to
operator-go v0.12.6) with Gen 3: a GenericReconciler driving one
HbaseRoleGroupHandler. internal/ drops from 3384 to ~1200 lines.
The dependency bump and the refactor are one commit because they are atomic:
v0.13.0 removes the BaseCluster API the old packages are built on, so neither
half compiles without the other.
What moves where:
- HbaseCluster implements common.ClusterInterface. GetSpec() bridges the
typed master/regionServer/restServer fields into the generic Roles map;
the status embeds GenericClusterStatus. The image adapter keeps the old
code-level defaults (repo, productVersion, kubedoopVersion) so the
resolved image reference is unchanged.
- DeclareRoles returns a RoleCatalog: ports, main container name, log
producers, the entrypoint (as Command), the probe set, and the default
affinity as ConfigDefaults. All of it is DECLARED before the framework
builds the StatefulSet, which is what keeps a user's podOverrides
outranking the product -- a post-build edit would invert that silently.
- ResolveRoleGroup contributes hbase-site.xml beneath the CRD overrides,
reading the ZooKeeper znode discovery ConfigMap through the client the
RoleGroupResolver seam now provides.
- Kerberos and the HDFS discovery mount flow through
buildCtx.VolumeProviders; the secret-operator CSI annotations are
preserved byte for byte.
- OIDC swaps the hand-built oauth2-proxy container for the framework's
native sidecar provider. The session cookie was derived from the CR UID,
which anyone able to read the CR could forge; it is now generated once
into <cluster>-oidc-cookie via reconciler.EnsureGeneratedSecret.
BREAKING CHANGE: the StatefulSet selector and the ServiceAccount name change,
and .spec.selector is immutable. An existing cluster must have its
StatefulSets deleted with --cascade=orphan and recreated by the new operator;
see docs/gen3-migration-notes.md for the full intentional-diff list.
BREAKING CHANGE: an OIDC-enabled cluster now requires Kubernetes 1.29+ (1.33
for GA). The oauth2-proxy sidecar carries probes on a native sidecar, which
older API servers reject -- the StatefulSet is created but never produces a
pod, visible only in its FailedCreate events.
Verified with the full chainsaw suite (default, kerberos, oidc, pdb,
observability) on Kubernetes 1.35.0; test/e2e/ is unchanged, so the suite
gates the migration rather than being adjusted to it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
KIND_K8S_VERSION defaulted to 1.26.15 while .github/workflows/test.yml runs 1.35.0, so `make setup-chainsaw-cluster` built a cluster CI never tests. That gap is no longer cosmetic: the operator injects oauth2-proxy as a native sidecar, and probes on a restartPolicy:Always init container are only accepted from Kubernetes 1.29. On 1.26 every pod of an OIDC-enabled cluster is rejected with "livenessProbe: Forbidden: may not be set for init containers" -- the StatefulSet is created, no pod ever is, nothing crash-loops, and the only trace is a FailedCreate event on the StatefulSet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…uced gen3-migration-notes.md carries the architecture mapping, the intentional-diff list (what the rendered resources gain or lose relative to Gen 2b, including the two breaking changes), the verification status, and three pre-existing defects left unfixed on purpose to keep byte parity with the e2e assertions. operator-go-issues.md holds the upstream findings this migration produced, each checked against the framework source: the default pod-affinity builder that four operators ship verbatim, the missing Gen 2b -> Gen 3 migration document, and an observation still to be confirmed -- the apply path reports a retryable write conflict as a hard error, so a normal delete and a routine StatefulSet status race both flip Degraded, the one condition the framework's own docs call the one worth alerting on. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Migrates hbase-operator from Gen 2b (
BaseCluster+ per-role packages, operator-go v0.12.6) to Gen 3: aGenericReconcilerdriving oneHbaseRoleGroupHandler, on the released operator-go v0.13.0.internal/drops from 3384 to ~1200 lines (-64%): thecluster/,master/,regionserver/,restserver/,common/andauthz/packages are replaced by a handler plus one cluster extension.How it is verified
The full chainsaw suite passes on Kubernetes 1.35.0 (the CI matrix version):
test/e2e/is unchanged. The suite gates the migration rather than being adjusted to it — and it earned that role twice, catching a Kubernetes-version incompatibility and an ordering bug that the unit tests did not.make lint(0 issues),make test(83% coverage oninternal/controller),go build,go vetall green. The CRD diff is additive only: the sole structural changes are theroleGroupsandobservedGenerationstatus fields; the 60 removed lines are alldefault:entries inside role config blocks, which upstream #544 removed from the commons API andapi/v1alpha1/crd_schema_test.gonow guards against.A new
internal/controller/handler_test.gorenders each role against a fake client and asserts the parity that matters: entrypoint, probe ports, volumes, metrics Service shape, Kerberos CSI annotations, and the OIDC sidecar.1. Existing clusters need manual StatefulSet recreation. The framework's selector labels and the derived ServiceAccount name differ from v0.12.6's, and
.spec.selectoris immutable. Upgrading an existing cluster fails in two stages, the first silent: the apply path preserves the live selector and emitsImmutableFieldIgnored, then the StatefulSet update is rejected because the pod template no longer matches. Operators must run:and let the new operator recreate them. Release notes must carry this.
2. OIDC now requires Kubernetes 1.29+ (1.33 for GA). oauth2-proxy moves to the framework's native sidecar, whose probes older API servers reject on an init container. The failure mode is quiet — the StatefulSet exists, no pod is ever created, nothing crash-loops, and the only trace is a
FailedCreateevent.Makefile'sKIND_K8S_VERSIONdefault (stale at 1.26.15, while CI ran 1.35.0) is corrected in its own commit for the same reason.Both, plus nine smaller rendering differences, are enumerated in
docs/gen3-migration-notes.md.Notable behaviour changes
getthe CR; it is now generated once into<cluster>-oidc-cookiethroughreconciler.EnsureGeneratedSecret, which fills a missing key without rotating the existing one. The user-facing credentials Secret keeps itsCLIENT_ID/CLIENT_SECRETcontract, so no CR changes.app.kubernetes.io/name: hbasewhile the pods carriedname: hbasecluster, so it never matched anything. Under Gen 3 the label ishbaseand the rule takes effect — worth knowing when scheduling multi-replica roles on small clusters.RoleGroupResolverseam, so userconfigOverridesstill win over every value the operator computes.Pre-existing defects left unfixed
Three are documented rather than fixed, to keep byte parity with the e2e assertions: the master metrics Service targets a named port that role does not declare;
HBASE_<role>_OPTSuses a lowercase role name HBase never reads; the kerberos discovery config was already dead code. Each needs its own change plus an e2e update.Upstream feedback produced
docs/operator-go-issues.mdrecords what this migration found in operator-go, each verified against framework source — chiefly the default pod-affinity builder that four operators ship byte-for-byte identically, and the absence of a Gen 2b → Gen 3 migration document, which every operator with running clusters will need. One earlier finding (ProductConfigcould not read the cluster its own docs described) was fixed upstream in #591 while this work was in progress, and the local workaround has been removed.🤖 Generated with Claude Code