Skip to content

[NE-2821] Implement e2e test for TLSAdherence on Gateway API#31415

Draft
rikatz wants to merge 2 commits into
openshift:mainfrom
rikatz:tls-gateway-config
Draft

[NE-2821] Implement e2e test for TLSAdherence on Gateway API#31415
rikatz wants to merge 2 commits into
openshift:mainfrom
rikatz:tls-gateway-config

Conversation

@rikatz

@rikatz rikatz commented Jul 22, 2026

Copy link
Copy Markdown
Member

Gateway API, implemented via Istio and Sail Library can now respect centralized TLS configuration from OpenShift.

This centralized configuration is implemented and respected with TLSAdherence set as strict.

The current change implements an e2e test for Gateway API when TLSAdherence is available and can be used by admins to also define that Gateways provisioned through OCP Gateway API must respect a specific TLS profile

Marking as draft while openshift/cluster-ingress-operator#1480 is not merged

Summary by CodeRabbit

  • New Features

    • Added end-to-end Gateway API TLS enforcement tests using externally reachable load balancers.
    • Validate Gateway listener TLS behavior against configured minimum and maximum TLS versions.
    • Expanded TLS reconciliation coverage to include additional Gateway-based TLS targets where supported.
  • Tests

    • Added automated Gateway provisioning with certificate-based HTTPS listeners and polling for readiness.
    • Assert TLS connection acceptance/rejection based on the allowed TLS version range.
    • Improved target reporting and reconciliation polling to incorporate dynamically added targets.

Gateway API, implemented via Istio and Sail Library can now respect centralized
TLS configuration from OpenShift.

This centralized configuration is implemented and respected with TLSAdherence
set as strict.

The current change implements an e2e test for Gateway API when TLSAdherence is available
and can be used by admins to also define that Gateways provisioned through OCP Gateway API
must respect a specific TLS profile
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 22, 2026
@openshift-ci

openshift-ci Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rikatz
Once this PR has been reviewed and has the lgtm label, please assign gangwgr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 6bbfc985-006d-47d9-98bd-e29a1494670f

📥 Commits

Reviewing files that changed from the base of the PR and between d8306b4 and 42baf75.

📒 Files selected for processing (2)
  • test/extended/tls/gateway_tls.go
  • test/extended/tls/tls_observed_config.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • test/extended/tls/gateway_tls.go
  • test/extended/tls/tls_observed_config.go

Walkthrough

Adds Gateway API LoadBalancer TLS validation with prerequisite checks, resource lifecycle handling, TLS probing, and integration into disruptive API server TLS profile reconciliation.

Changes

Gateway TLS adherence validation

Layer / File(s) Summary
Gateway eligibility and resource provisioning
test/extended/tls/gateway_tls.go
Checks feature, platform, and service-network prerequisites; creates or reuses Gateway resources, waits for programming and an external address, and performs best-effort cleanup.
Gateway TLS probing
test/extended/tls/gateway_tls.go
Connects to the external LoadBalancer address and verifies accepted and rejected TLS configurations.
TLS reconciliation integration
test/extended/tls/tls_observed_config.go
Adds extra TLS targets, conditionally provisions the Gateway target, sets TLS adherence policy, and includes the target in reconciliation and validation.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TLSConfigTest
  participant GatewayAPI
  participant GatewayController
  participant LoadBalancer
  participant gatewayLBTarget
  TLSConfigTest->>GatewayAPI: Create GatewayClass, Secret, and HTTPS Gateway
  GatewayController-->>GatewayAPI: Set Programmed=True
  GatewayAPI->>LoadBalancer: Provision external address
  LoadBalancer-->>TLSConfigTest: Return hostname or IP
  TLSConfigTest->>gatewayLBTarget: Add Gateway TLS target
  gatewayLBTarget->>LoadBalancer: Test accepted and rejected TLS profiles
  LoadBalancer-->>gatewayLBTarget: Return TLS success or failure
Loading
🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Ipv6 And Disconnected Network Test Compatibility ⚠️ Warning New Gateway TLS e2e skips IPv6/dual-stack service networks and builds address:port without net.JoinHostPort, so it isn’t IPv6-safe. Verify on /payload-job periodic-ci-openshift-release-master-nightly-4.22-e2e-metal-ipi-ovn-ipv6; use net.JoinHostPort for the LB dial and avoid skipping IPv6/dual-stack.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: adding an end-to-end TLSAdherence test for Gateway API.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All Ginkgo titles in the changed TLS tests are static string literals; no dynamic/generated values appear in It/Describe/Context/When names.
Test Structure And Quality ✅ Passed Gateway resources are cleaned up via fixture defer, cluster waits use explicit timeouts, and the new target follows existing gateway test patterns.
Microshift Test Compatibility ✅ Passed The new TLSObservedConfig test suite is guarded by exutil.IsMicroShiftCluster()+g.Skip in BeforeAll, so the added FeatureGate/ClusterVersion/Gateway API usage won't run on MicroShift.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The new TLS/Gateway checks only use a single Gateway LoadBalancer and no node-count, scheduling, drain, or HA assumptions; no SNO-specific skip is needed.
Topology-Aware Scheduling Compatibility ✅ Passed Changes are test-only and add no node selectors, affinity, spread constraints, replicas, or PDBs; the Gateway test only creates API objects and polls status.
Ote Binary Stdout Contract ✅ Passed No process-level stdout writes found: touched files only use e2e.Logf inside suite/spec code; no fmt.Print, klog, log.SetOutput, init, or TestMain paths.
No-Weak-Crypto ✅ Passed No MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or non-constant-time secret/token comparisons were found; certgen uses standard ECDSA P-256 and x509.
Container-Privileges ✅ Passed Changed TLS e2e code only creates Gateway/Secret resources; scans found no privileged/securityContext fields, SYS_ADMIN, hostNetwork/PID/IPC, or runAsUser=0.
No-Sensitive-Data-In-Logs ✅ Passed Logs only show resource names, TLS profiles, and LB host:port; I found no passwords, tokens, PII, or secret contents exposed.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/extended/tls/tls_observed_config.go (1)

1296-1313: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Restore APIServer adherence before the test finishes.

When Gateway API TLS validation runs, this test sets APIServer.spec.tlsAdherence = StrictAllComponents but leaves no cleanup path. Since tlsAdherence cannot be unset once set, re-running the same disruptive test or another test that needs a different adherence value on the same cluster can hit a permanent API validation error instead of a retryable 409 conflict. Capture and restore the compliance-affected adherence value, or avoid modifying the immutable field unless absolutely required.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/tls/tls_observed_config.go` around lines 1296 - 1313, Update
the test cleanup around setAPIServerTLSProfile so any tlsAdherence value changed
to StrictAllComponents is captured before modification and restored before the
test completes. Preserve the existing NoOpinion behavior for callers that do not
need adherence changes, and ensure restoration uses the same APIServer update
path with conflict retry.
🧹 Nitpick comments (2)
test/extended/tls/gateway_tls.go (2)

230-256: 🔒 Security & Privacy | 🔵 Trivial | 💤 Low value

Ephemeral test private key isn't explicitly zeroized after use.

privateKey/pemKey hold the private key material for the self-signed test cert and are only released via GC. The crypto-security guideline for files matching *tls* calls for zeroizing key material rather than relying on the garbage collector. Given this is a throwaway, non-sensitive test certificate, the practical risk is minimal, but flagging for guideline compliance.

As per path instructions, "Zeroize key material after use (no garbage-collector reliance)."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/tls/gateway_tls.go` around lines 230 - 256, Explicitly zeroize
the private key material in the certificate-generation flow after it is no
longer needed, including the `privateKey` and any mutable key bytes represented
by `pemKey`, while preserving the existing Secret creation behavior and ensuring
cleanup occurs on error paths as well.

Source: Path instructions


125-134: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

Consider tls.Dialer.DialContext instead of tls.DialWithDialer.

DialWithDialer uses context.Background() internally and only respects the dialer's fixed 10s timeout; it ignores the ctx parameter already passed into testTLS. Switching to (&tls.Dialer{NetDialer: dialer, Config: cfg}).DialContext(ctx, "tcp", hostPort) would let the dial respect the caller's overall test deadline/cancellation in addition to the per-dial timeout.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/extended/tls/gateway_tls.go` around lines 125 - 134, Update testTLS to
use a tls.Dialer configured with the existing dialer and expected TLS
configuration, then call DialContext with the provided ctx, "tcp", and hostPort.
Preserve the existing connection error reporting and close the returned
connection as before.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/tls/tls_observed_config.go`:
- Around line 512-521: Ensure the gatewayFixture returned by
setupGatewayTLSTarget is cleaned up regardless of whether setup succeeds or
returns an error. Register gatewayFixture.cleanup(ctx) immediately after
setupGatewayTLSTarget returns, before branching on err, while preserving the
existing error logging and successful target registration behavior.

---

Outside diff comments:
In `@test/extended/tls/tls_observed_config.go`:
- Around line 1296-1313: Update the test cleanup around setAPIServerTLSProfile
so any tlsAdherence value changed to StrictAllComponents is captured before
modification and restored before the test completes. Preserve the existing
NoOpinion behavior for callers that do not need adherence changes, and ensure
restoration uses the same APIServer update path with conflict retry.

---

Nitpick comments:
In `@test/extended/tls/gateway_tls.go`:
- Around line 230-256: Explicitly zeroize the private key material in the
certificate-generation flow after it is no longer needed, including the
`privateKey` and any mutable key bytes represented by `pemKey`, while preserving
the existing Secret creation behavior and ensuring cleanup occurs on error paths
as well.
- Around line 125-134: Update testTLS to use a tls.Dialer configured with the
existing dialer and expected TLS configuration, then call DialContext with the
provided ctx, "tcp", and hostPort. Preserve the existing connection error
reporting and close the returned connection as before.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 14ea05da-bef0-4fd0-b1f2-52e8ff5ddea2

📥 Commits

Reviewing files that changed from the base of the PR and between c367745 and d8306b4.

📒 Files selected for processing (2)
  • test/extended/tls/gateway_tls.go
  • test/extended/tls/tls_observed_config.go

Comment thread test/extended/tls/tls_observed_config.go
@openshift-ci openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Jul 22, 2026
@gcs278

gcs278 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

/test ?

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@gcs278: The following commands are available to trigger required jobs:

/test agentic-images
/test e2e-aws-csi
/test e2e-aws-jenkins
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-image-registry
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-builds
/test e2e-gcp-ovn-image-ecosystem
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi
/test go-verify-deps
/test images
/test lint
/test okd-scos-images
/test unit
/test verify
/test verify-deps
/test verify-image-manifest-lists

The following commands are available to trigger optional jobs:

/test e2e-agent-compact-ipv4-iso-no-registry
/test e2e-agnostic-ovn-cmd
/test e2e-aws-disruptive
/test e2e-aws-etcd-certrotation
/test e2e-aws-etcd-recovery
/test e2e-aws-ovn
/test e2e-aws-ovn-cgroupsv2
/test e2e-aws-ovn-dra-example
/test e2e-aws-ovn-edge-zones
/test e2e-aws-ovn-etcd-scaling
/test e2e-aws-ovn-kube-apiserver-rollout
/test e2e-aws-ovn-kubevirt
/test e2e-aws-ovn-serial-fast
/test e2e-aws-ovn-serial-ipsec
/test e2e-aws-ovn-serial-publicnet-1of2
/test e2e-aws-ovn-serial-publicnet-2of2
/test e2e-aws-ovn-single-node
/test e2e-aws-ovn-single-node-serial
/test e2e-aws-ovn-single-node-techpreview
/test e2e-aws-ovn-single-node-techpreview-serial
/test e2e-aws-ovn-single-node-upgrade
/test e2e-aws-ovn-upgrade
/test e2e-aws-ovn-upgrade-rollback
/test e2e-aws-ovn-upi
/test e2e-aws-proxy
/test e2e-aws-tls-observed-config
/test e2e-aws-tls-observed-config-fips
/test e2e-aws-tls-observed-config-hypershift
/test e2e-azure
/test e2e-azure-ovn-etcd-scaling
/test e2e-azure-ovn-upgrade
/test e2e-baremetalds-kubevirt
/test e2e-external-aws
/test e2e-external-aws-ccm
/test e2e-external-vsphere-ccm
/test e2e-gcp-disruptive
/test e2e-gcp-fips-serial-1of2
/test e2e-gcp-fips-serial-2of2
/test e2e-gcp-ovn-etcd-scaling
/test e2e-gcp-ovn-kube-apiserver-rollout
/test e2e-gcp-ovn-rt-upgrade
/test e2e-gcp-ovn-techpreview
/test e2e-gcp-ovn-techpreview-pkiconfig
/test e2e-gcp-ovn-techpreview-serial-1of2
/test e2e-gcp-ovn-techpreview-serial-2of2
/test e2e-gcp-ovn-usernamespace
/test e2e-hypershift-conformance
/test e2e-metal-ipi-ovn
/test e2e-metal-ipi-ovn-bgp-virt-dualstack
/test e2e-metal-ipi-ovn-bgp-virt-dualstack-techpreview
/test e2e-metal-ipi-ovn-dualstack
/test e2e-metal-ipi-ovn-dualstack-bgp
/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
/test e2e-metal-ipi-ovn-dualstack-local-gateway
/test e2e-metal-ipi-ovn-kube-apiserver-rollout
/test e2e-metal-ipi-serial-1of2
/test e2e-metal-ipi-serial-2of2
/test e2e-metal-ipi-serial-ovn-ipv6-1of2
/test e2e-metal-ipi-serial-ovn-ipv6-2of2
/test e2e-metal-ipi-virtualmedia
/test e2e-metal-ovn-single-node-live-iso
/test e2e-metal-ovn-single-node-with-worker-live-iso
/test e2e-metal-ovn-two-node-arbiter
/test e2e-metal-ovn-two-node-fencing
/test e2e-metal-ovn-two-node-fencing-recovery
/test e2e-openstack-dualstack-v6primary
/test e2e-openstack-ovn
/test e2e-openstack-serial
/test e2e-vsphere-ovn-etcd-scaling
/test okd-scos-e2e-aws-ovn

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-origin-main-agentic-images
pull-ci-openshift-origin-main-go-verify-deps
pull-ci-openshift-origin-main-images
pull-ci-openshift-origin-main-lint
pull-ci-openshift-origin-main-okd-scos-images
pull-ci-openshift-origin-main-unit
pull-ci-openshift-origin-main-verify
pull-ci-openshift-origin-main-verify-deps
Details

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@rikatz

rikatz commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/origin/main/e2e-gcp-ovn-techpreview-serial-1of2 openshift/cluster-ingress-operator#1480

@rikatz

rikatz commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/origin/main/e2e-gcp-ovn-techpreview-serial-2of2 openshift/cluster-ingress-operator#1480

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@rikatz, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-gcp-ovn-techpreview-serial-2of2' not found in injected config

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@rikatz, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-gcp-ovn-techpreview-serial-1of2' not found in injected config

@rikatz

rikatz commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/testwith e2e-gcp-ovn-techpreview-serial-1of2 openshift/cluster-ingress-operator#1480

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@rikatz, testwith: Error processing request. ERROR:

could not determine job runs: requested job is invalid. needs to be formatted like: <org>/<repo>/<branch>/<variant?>/<job>. instead it was: e2e-gcp-ovn-techpreview-serial-1of2

@rikatz

rikatz commented Jul 23, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/origin/main/e2e-gcp-ovn-techpreview-serial-1of2 openshift/cluster-ingress-operator#1480

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@rikatz, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-gcp-ovn-techpreview-serial-1of2' not found in injected config

@gcs278

gcs278 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

/testwith openshift/origin/main/e2e-gcp-ovn-techpreview-serial-1of2 openshift/cluster-ingress-operator#1480

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@gcs278, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-gcp-ovn-techpreview-serial-1of2' not found in injected config

@gcs278

gcs278 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

/testwith openshift/origin/main/e2e-gcp-ovn openshift/api#2883 openshift/cluster-ingress-operator#1480

@gcs278

gcs278 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

/testwith openshift/origin/main/e2e-aws-ovn-serial-1of2 openshift/api#2883 openshift/cluster-ingress-operator#1480

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@gcs278, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-aws-ovn-serial-1of2' not found in injected config

@gcs278

gcs278 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

/testwith openshift/origin/main/e2e-aws-ovn-serial-2of2 openshift/api#2883 openshift/cluster-ingress-operator#1480

@openshift-ci

openshift-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@gcs278, testwith: could not generate prow job. ERROR:

BUG: test 'e2e-aws-ovn-serial-2of2' not found in injected config

@gcs278

gcs278 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

/testwith openshift/origin/main/e2e-gcp-ovn-techpreview-serial openshift/cluster-ingress-operator#1480

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants