From 7b784cd8394e9105d5477f92346a02dab328bf6e Mon Sep 17 00:00:00 2001 From: Sandhya Dasu Date: Thu, 31 Aug 2023 17:18:09 -0400 Subject: [PATCH] Customer configured DNS for cloud platforms AWS, Azure and GCP --- ...{aws-custom-dns.md => cloud-custom-dns.md} | 153 ++++++++++-------- 1 file changed, 87 insertions(+), 66 deletions(-) rename enhancements/installer/{aws-custom-dns.md => cloud-custom-dns.md} (92%) diff --git a/enhancements/installer/aws-custom-dns.md b/enhancements/installer/cloud-custom-dns.md similarity index 92% rename from enhancements/installer/aws-custom-dns.md rename to enhancements/installer/cloud-custom-dns.md index f7de151d717..6c2e92d072f 100644 --- a/enhancements/installer/aws-custom-dns.md +++ b/enhancements/installer/cloud-custom-dns.md @@ -1,5 +1,5 @@ --- -title: custom-dns +title: cloud-custom-dns authors: - "@sadasu" reviewers: @@ -63,11 +63,16 @@ on public cloud providers. - As an administrator, I want to continue using the LB services provided by the underlying cloud platform. +- As a user running their cluster on AWS GovCloud, I would like my cluster to +be publicly accessible. Currently, with Route53, only private clusters can be +created in AWS GovCloud. + + ### Goals - Enable AWS, Azure, and GCP customers to use their custom DNS solution in place of the cloud solution (For example, Route53 for AWS). -- Provid in-cluster DNS solution for successful cluster installation without +- Provide in-cluster DNS solution for successful cluster installation without dependence on customer configured infrastructure items. - Continue using the cloud based LB service for API, Ingress and API-Int. @@ -92,10 +97,9 @@ DNS solutions [Route53 for AWS](https://aws.amazon.com/route53/), [Azure DNS](ht The Installer configures the LBs for the API and API-Int services and the Ingress Controllers configure the LBs for the *.apps service. There is -currently no way of knowing these LB IP addresses before their creation. So, -the customer would have to wait to configure their custom DNS solution until -after the LBs are created by OpenShift and the cluster installation has -completed. +currently no way of knowing these LB IP addresses before their creation. The +customer would have to wait to configure their custom DNS solution until after +the LBs are created by OpenShift and the cluster installation has completed. For the cluster installation to succeed before the custom DNS solution is setup for `api`, `api-int` and `*.apps` resolution, OpenShift will have to provide a @@ -194,57 +198,29 @@ this information to generate the CoreDNS CoreFile. ### API Extensions -1. The AWSPlatformStatus within the PlatformStatus field of the Infrastructure -ConfigResource (CR) is updated to contain all the DNS config required for the -in-cluster CoreDNS solution. This same CR is available to the user post a -successful cluster install, to configure their own DNS solution. - -```go -type AWSPlatformStatus struct { - - // AWSClusterDNSConfig contains all the DNS config required to configure a custom DNS solution. - // +optional - AWSClusterDNSConfig *ClusterDNSConfig `json:"awsClusterDNSConfig,omitempty"` - - - -} - -type ClusterDNSConfig struct { - // APIServerDNSConfig contains information to configure DNS for API Server. - // This field will be set only when the userConfiguredDNS feature is enabled. - APIServerDNSConfig []DNSConfig `json:"apiServerDNSConfig,omitempty"` - - // InternalAPIServerDNSConfig contains information to configure DNS for the Internal API Server. - // This field will be set only when the userConfiguredDNS feature is enabled. - InternalAPIServerDNSConfig []DNSConfig `json:"internalAPIServerDNSConfig,omitempty"` - - // IngressDNSConfig contains information to configure DNS for cluster services. - // This field will be set only when the userConfiguredDNS feature is enabled. - IngressDNSConfig []DNSConfig `json:"ingressDNSConfig,omitempty"` -} - - -type DNSConfig struct { - // recordType is the DNS record type. - RecordType string `json:"recordType"` - - // lBIPAddress is the Load Balancer IP address for DNS config - LBIPAddress string `json:"lbIPAddress"` -} +1. A new ConfigMap called `lbConfigForDNS` is created by the Installer. It can +be created in any namespace but we are choosing to create it in the same +namespace as the CoreDNS pods. Hence, the namespace name would be constructed +as: openshift-$platform_name-infra +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: LBConfigForDNS + namespace: openshift-aws-infra +data: + internal-api-lb-dns-name: "abc-123" + external-api-lb-dns-name: "xyz-456" ``` 2. Install config is updated to allow the customer to specify if an external -user configured DNS will be used. `UserConfiguredDNS` is added to the -install-config and will have to be explicitly set to `Enabled` to enable this -functionality. This config is not added to any platform specific section of -the config because there are plans to allow this functionality in Azure and GCP -too. The validation for this config will disallow this value being `Enabled` in -platforms that currently do not support it. +user configured DNS will be used. `UserConfiguredDNS` is added to the platform +portions of the install-config. The useer will have to be explicitly set it to +`Enabled` to enable this functionality. This field is added to the AWS, Azure +and GCP platforms. ```yaml ---- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: @@ -264,6 +240,8 @@ spec: - name: v1 schema: + aws/azure/gcp: + properties: userConfiguredDNS: description: UserConfiguredDNS is set to `Enabled` when the customer wants to use a DNS solution external to the cluster and OpenShift is @@ -276,27 +254,70 @@ spec: type: string ``` -3. Add a field within the `PlatformSpec` for AWS, Azure and GCP to indicate if -custom DNS is enabled. `PlatformSpec` is within the `Spec` field of the -Infrastructure CR. Here is the update for platform AWS. +3. Once this feature is enabled, we don't expect it to change day 2. For that +reason, add a field within the `PlatformStatus` for AWS, Azure and GCP to +indicate if custom DNS is enabled. `PlatformStatus` is within the `Status` +field of the Infrastructure CR. + +Here is the update for platform AWS. ```yaml // AWSPlatformSpec holds the desired state of the Amazon Web Services infrastructure provider. // This only includes fields that can be modified in the cluster. -type AWSPlatformSpec struct { - // serviceEndpoints list contains custom endpoints which will override default - // service endpoint of AWS Services. - // There must be only one ServiceEndpoint for a service. - // +optional - ServiceEndpoints []AWSServiceEndpoint `json:"serviceEndpoints,omitempty"` - - // customDNS indicates if the customer is providing their own DNS +type AWSPlatformStatus struct { + + + // userProvisionedDNS indicates if the customer is providing their own DNS // solution inplace of the default provided by the underlying platform. - // Its value is set by the Installer and can have 2 possible values: - // "Enabled" and "Disabled" with "Disabled" being the default. - CustomDNS string ``json:"customDNS,omitempty"` + // Its value is set by the Installer and can have 2 possible values: + // "Enabled" and "Disabled" with "Disabled" being the default. + UserProvisionedDNS string ``json:"userProvisionedDNS,omitempty"` } ``` + +4. Optional: The AWSPlatformStatus within the PlatformStatus field of the Infrastructure +ConfigResource (CR) is updated to contain all the DNS config required for the +in-cluster CoreDNS solution. This same CR is available to the user post a +successful cluster install, to configure their own DNS solution. + +```go +type AWSPlatformStatus struct { + + // AWSClusterDNSConfig contains all the DNS config required to configure a custom DNS solution. + // +optional + AWSClusterDNSConfig *ClusterDNSConfig `json:"awsClusterDNSConfig,omitempty"` + + + +} + +type ClusterDNSConfig struct { + // APIServerDNSConfig contains information to configure DNS for API Server. + // This field will be set only when the userConfiguredDNS feature is enabled. + APIServerDNSConfig []DNSConfig `json:"apiServerDNSConfig,omitempty"` + + // InternalAPIServerDNSConfig contains information to configure DNS for the Internal API Server. + // This field will be set only when the userConfiguredDNS feature is enabled. + InternalAPIServerDNSConfig []DNSConfig `json:"internalAPIServerDNSConfig,omitempty"` + + // IngressDNSConfig contains information to configure DNS for cluster services. + // This field will be set only when the userConfiguredDNS feature is enabled. + IngressDNSConfig []DNSConfig `json:"ingressDNSConfig,omitempty"` +} + + +type DNSConfig struct { + // recordType is the DNS record type. + RecordType string `json:"recordType"` + + // lBIPAddress is the Load Balancer IP address for DNS config + LBIPAddress string `json:"lbIPAddress"` +} + +``` +Optional because the user can be asked to look at the `LBConfigForDNS` +config map for the same information. + ### Implementation Details/Notes/Constraints [optional] @@ -313,7 +334,7 @@ customer's configuration of their custom DNS solution. Today, the Installer configures the LB first and the IP address of the LB is used to configure DNS records for the API, API-Int and `*.apps`. Since the IPs -cannot be predicted in advance, configing the customer's external DNS before +cannot be predicted in advance, configuring the customer's external DNS before cluster install is not a possibility. Then we have the option of configuring the LB manually(by the customer) and