test.csv

Re-evaluated workload endpoint status adminUp true failed false known true operUp true status up workloadEndpointID proto.WorkloadEndpointID OrchestratorId k8s WorkloadId cluster-614da81d67f5a62e81967366/cluster-management-agent-775bb9bc5-vm2mr EndpointId eth0
Watch error received from Upstream ListRoot /calico/ipam/v2/host/ip-10-0-1-75.ec2.internal error too old resource version 426507518 492092109
Couldn t reconcile node routes error listing routes unable to find route table for AWS cluster prod1
Re-evaluated workload endpoint status adminUp false failed false known false operUp false status workloadEndpointID proto.WorkloadEndpointID OrchestratorId k8s WorkloadId hubble-system/packsync-1634403600-9mpsq EndpointId eth0
<DATE> <TIME> W | etcdserver failed to send out heartbeat on time exceeded the 100ms timeout for <*> to <*>
error retrieving resource lock <*> Get https //<IP> <*> 10s net/http request canceled Client.Timeout exceeded while awaiting headers
Failed to update SpectroClusterStatusConditions Code ResourceNotFound Msg Cluster 615c32c6cc841e9ea490508e is not found Ref 3a47b86f453ac3e02af481740b329932dee1d539 Details <nil> cluster ehs20testdeploy-5-10/615c32c6cc841e9ea490508e/cleanup
Failed to push status conditions & Code ResourceNotFound Details <nil> Message Cluster 615c32c6cc841e9ea490508e is not found Ref 3a47b86f453ac3e02af481740b329932dee1d539 cluster ehs20testdeploy-5-10/615c32c6cc841e9ea490508e/cleanup
error deleting network for AWSManagedControlPlane error failed to delete vpc vpc-0d11162bb433f2453 DependencyViolation The vpc vpc-0d11162bb433f2453 has dependencies and cannot be deleted. n tstatus code 400 request id 3507d0cd-8b04-4566-9377-47c4996acde8 name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Failed to watch *v1.ConfigMap failed to list *v1.ConfigMap Get https //<IP> 6443/api/v1/namespaces/kube-system/configmaps?fieldSelector metadata.name%3Dextension-apiserver-authentication&limit 500&resourceVersion 0 dial tcp <IP> 6443 connect connection refused
Failed to process spectro cluster status event & Code ResourceNotFound Details <nil> Message Cluster 615c32c6cc841e9ea490508e is not found Ref 3a47b86f453ac3e02af481740b329932dee1d539 cluster ehs20testdeploy-5-10/615c32c6cc841e9ea490508e/cleanup
Reconciler error error failed to delete vpc vpc-0d11162bb433f2453 DependencyViolation The vpc vpc-0d11162bb433f2453 has dependencies and cannot be deleted. n tstatus code 400 request id 3507d0cd-8b04-4566-9377-47c4996acde8 controller awsmanagedcontrolplane name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Failed to watch <*> failed to list <*> Get https //<IP> <*> 500&resourceVersion 0 dial tcp <IP> 6443 connect connection refused
error deleting network for AWSManagedControlPlane error error patching conditions The condition ClusterSecurityGroupsReady was modified by a different process and this caused a merge/ChangeCondition conflict &v1alpha3.Condition n tType ClusterSecurityGroupsReady n tStatus False n tSeverity Info n- tLastTransitionTime v1.Time Time s <DATE> <TIME> UTC n+ tLastTransitionTime v1.Time Time s <DATE> <TIME> UTC n- tReason Deleting n+ tReason Deleted n tMessage n n name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Reconciler error error error patching conditions The condition ClusterSecurityGroupsReady was modified by a different process and this caused a merge/ChangeCondition conflict &v1alpha3.Condition n tType ClusterSecurityGroupsReady n tStatus False n tSeverity Info n- tLastTransitionTime v1.Time Time s <DATE> <TIME> UTC n+ tLastTransitionTime v1.Time Time s <DATE> <TIME> UTC n- tReason Deleting n+ tReason Deleted n tMessage n n controller awsmanagedcontrolplane name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Calico Node referenced in IPAM data does not exist error resource does not exist Node ip-10-0-1-5.ec2.internal with error nodes ip-10-0-1-5.ec2.internal not found
Failed to watch <*> failed to list <*> Get https //<IP> <*> 500&resourceVersion 0 net/http TLS handshake timeout
Failed to watch *v1.ConfigMap failed to list *v1.ConfigMap Get https //<IP> 6443/api/v1/namespaces/kube-system/configmaps?fieldSelector metadata.name%3Dextension-apiserver-authentication&limit 500&resourceVersion 0 net/http TLS handshake timeout
Re-evaluated workload endpoint status adminUp true failed false known true operUp false status down workloadEndpointID proto.WorkloadEndpointID OrchestratorId k8s WorkloadId <*> EndpointId eth0
error deleting network for AWSManagedControlPlane error error patching conditions The condition ClusterSecurityGroupsReady was modified by a different process and this caused a merge/ChangeCondition conflict &v1alpha3.Condition n t... // 2 identical fields n tSeverity Info n tLastTransitionTime Time s <DATE> <TIME> UTC n- tReason Deleting n+ tReason Deleted n tMessage n n name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Reconciler error error error patching conditions The condition ClusterSecurityGroupsReady was modified by a different process and this caused a merge/ChangeCondition conflict &v1alpha3.Condition n t... // 2 identical fields n tSeverity Info n tLastTransitionTime Time s <DATE> <TIME> UTC n- tReason Deleting n+ tReason Deleted n tMessage n n controller awsmanagedcontrolplane name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Successfully loaded configuration. GOMAXPROCS 4 builddate <DATETIME>+0000 config &config.Config UseInternalDataplaneDriver true DataplaneDriver calico-iptables-plugin WireguardEnabled false WireguardListeningPort 51820 WireguardRoutingRulePriority 99 WireguardInterfaceName wireguard.cali WireguardMTU 0 BPFEnabled false BPFDisableUnprivileged true BPFLogLevel off BPFDataIfacePattern *regexp.Regexp <HEX> BPFConnectTimeLoadBalancingEnabled true BPFExternalServiceMode tunnel BPFKubeProxyIptablesCleanupEnabled true BPFKubeProxyMinSyncPeriod 1000000000 BPFKubeProxyEndpointSlicesEnabled false BPFExtToServiceConnmark 0 DebugBPFCgroupV2 DebugBPFMapRepinEnabled true DatastoreType kubernetes FelixHostname ip-10-0-5-116.ec2.internal EtcdAddr <IP> 2379 EtcdScheme http EtcdKeyFile EtcdCertFile EtcdCaFile EtcdEndpoints string nil TyphaAddr TyphaK8sServiceName TyphaK8sNamespace kube-system TyphaReadTimeout 30000000000 TyphaWriteTimeout 10000000000 TyphaKeyFile TyphaCertFile TyphaCAFile TyphaCN TyphaURISAN Ipv6Support false IptablesBackend auto RouteRefreshInterval 90000000000 InterfaceRefreshInterval 90000000000 DeviceRouteSourceAddress net.IP nil DeviceRouteProtocol 3 RemoveExternalRoutes true IptablesRefreshInterval 90000000000 IptablesPostWriteCheckIntervalSecs 1000000000 IptablesLockFilePath /run/xtables.lock IptablesLockTimeoutSecs 0 IptablesLockProbeIntervalMillis 50000000 FeatureDetectOverride map string string nil IpsetsRefreshInterval 10000000000 MaxIpsetSize 1048576 XDPRefreshInterval 90000000000 PolicySyncPathPrefix NetlinkTimeoutSecs 10000000000 MetadataAddr MetadataPort 8775 OpenstackRegion InterfacePrefix cali InterfaceExclude *regexp.Regexp *regexp.Regexp <HEX> ChainInsertMode insert DefaultEndpointToHostAction ACCEPT IptablesFilterAllowAction ACCEPT IptablesMangleAllowAction ACCEPT LogPrefix calico-packet LogFilePath LogSeverityFile LogSeverityScreen INFO LogSeveritySys VXLANEnabled false VXLANPort 4789 VXLANVNI 4096 VXLANMTU 0 IPv4VXLANTunnelAddr net.IP nil VXLANTunnelMACAddr IpInIpEnabled true IpInIpMtu 0 IpInIpTunnelAddr net.IP <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> AllowVXLANPacketsFromWorkloads false AllowIPIPPacketsFromWorkloads false AWSSrcDstCheck DoNothing ServiceLoopPrevention Drop ReportingIntervalSecs 0 ReportingTTLSecs 90000000000 EndpointReportingEnabled false EndpointReportingDelaySecs 1000000000 IptablesMarkMask <HEX> DisableConntrackInvalidCheck false HealthEnabled true HealthPort 9099 HealthHost localhost PrometheusMetricsEnabled false PrometheusMetricsHost PrometheusMetricsPort 9091 PrometheusGoMetricsEnabled true PrometheusProcessMetricsEnabled true FailsafeInboundHostPorts config.ProtoPort config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol udp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> FailsafeOutboundHostPorts config.ProtoPort config.ProtoPort Net Protocol udp Port <HEX> config.ProtoPort Net Protocol udp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> KubeNodePortRanges numorstring.Port numorstring.Port MinPort <HEX> MaxPort <HEX> PortName NATPortRange numorstring.Port MinPort <HEX> MaxPort <HEX> PortName NATOutgoingAddress net.IP nil UsageReportingEnabled true UsageReportingInitialDelaySecs 300000000000 UsageReportingIntervalSecs 86400000000000 ClusterGUID f204e0a79d8246e78cc0f3765274d4c0 ClusterType k8s bgp kdd kubeadm CalicoVersion v3.19.0 ExternalNodesCIDRList string nil DebugMemoryProfilePath DebugCPUProfilePath /tmp/felix-cpu-<timestamp>.pprof DebugDisableLogDropping false DebugSimulateCalcGraphHangAfter 0 DebugSimulateDataplaneHangAfter 0 DebugPanicAfter 0 DebugSimulateDataRace false RouteSource CalicoIPAM RouteTableRange idalloc.IndexRange Min 1 Max 250 IptablesNATOutgoingInterfaceFilter SidecarAccelerationEnabled false XDPEnabled true GenericXDPEnabled false Variant Calico MTUIfacePattern *regexp.Regexp <HEX> internalOverrides map string string sourceToRawConfig map config.Source map string string <HEX> map string string CalicoVersion v3.19.0 ClusterGUID f204e0a79d8246e78cc0f3765274d4c0 ClusterType k8s bgp kdd kubeadm IpInIpEnabled true LogSeverityScreen Info ReportingIntervalSecs 0 <HEX> map string string IpInIpTunnelAddr <IP> <HEX> map string string LogFilePath None LogSeverityFile None LogSeveritySys None MetadataAddr None <HEX> map string string datastoretype kubernetes defaultendpointtohostaction ACCEPT felixhostname ip-10-0-5-116.ec2.internal healthenabled true ipinipmtu 0 ipv6support false rawValues map string string CalicoVersion v3.19.0 ClusterGUID f204e0a79d8246e78cc0f3765274d4c0 ClusterType k8s bgp kdd kubeadm DatastoreType kubernetes DefaultEndpointToHostAction ACCEPT FelixHostname ip-10-0-5-116.ec2.internal HealthEnabled true IpInIpEnabled true IpInIpMtu 0 IpInIpTunnelAddr <IP> Ipv6Support false LogFilePath None LogSeverityFile None LogSeverityScreen Info LogSeveritySys None MetadataAddr None ReportingIntervalSecs 0 Err error nil loadClientConfigFromEnvironment func *apiconfig.CalicoAPIConfig error <HEX> useNodeResourceUpdates false gitcommit 369b28f353621fb7a9b88baca6ee8ea43fb19733 version v3.19.0
Creating internal dataplane driver. config intdataplane.Config Hostname ip-10-0-5-116.ec2.internal IPv6Enabled false RuleRendererOverride rules.RuleRenderer nil IPIPMTU 0 VXLANMTU 0 VXLANPort 4789 MaxIPSetSize 1048576 IptablesBackend auto IPSetsRefreshInterval 10000000000 RouteRefreshInterval 90000000000 DeviceRouteSourceAddress net.IP nil DeviceRouteProtocol 3 RemoveExternalRoutes true IptablesRefreshInterval 90000000000 IptablesPostWriteCheckInterval 1000000000 IptablesInsertMode insert IptablesLockFilePath /run/xtables.lock IptablesLockTimeout 0 IptablesLockProbeInterval 50000000 XDPRefreshInterval 90000000000 Wireguard wireguard.Config Enabled false ListeningPort 51820 FirewallMark 0 RoutingRulePriority 99 RoutingTableIndex 1 InterfaceName wireguard.cali MTU 0 NetlinkTimeout 10000000000 RulesConfig rules.Config IPSetConfigV4 *ipsets.IPVersionConfig <HEX> IPSetConfigV6 *ipsets.IPVersionConfig <HEX> WorkloadIfacePrefixes string cali IptablesMarkAccept <HEX> IptablesMarkPass <HEX> IptablesMarkScratch0 <HEX> IptablesMarkScratch1 <HEX> IptablesMarkEndpoint <HEX> IptablesMarkNonCaliEndpoint <HEX> KubeNodePortRanges numorstring.Port numorstring.Port MinPort <HEX> MaxPort <HEX> PortName KubeIPVSSupportEnabled false OpenStackMetadataIP net.IP nil OpenStackMetadataPort <HEX> OpenStackSpecialCasesEnabled false VXLANEnabled false VXLANPort 4789 VXLANVNI 4096 IPIPEnabled true IPIPTunnelAddress net.IP <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> <HEX> VXLANTunnelAddress net.IP nil AllowVXLANPacketsFromWorkloads false AllowIPIPPacketsFromWorkloads false WireguardEnabled false WireguardInterfaceName wireguard.cali IptablesLogPrefix calico-packet EndpointToHostAction ACCEPT IptablesFilterAllowAction ACCEPT IptablesMangleAllowAction ACCEPT FailsafeInboundHostPorts config.ProtoPort config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol udp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> FailsafeOutboundHostPorts config.ProtoPort config.ProtoPort Net Protocol udp Port <HEX> config.ProtoPort Net Protocol udp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> config.ProtoPort Net Protocol tcp Port <HEX> DisableConntrackInvalid false NATPortRange numorstring.Port MinPort <HEX> MaxPort <HEX> PortName IptablesNATOutgoingInterfaceFilter NATOutgoingAddress net.IP nil BPFEnabled false ServiceLoopPrevention Drop IfaceMonitorConfig ifacemonitor.Config InterfaceExcludes *regexp.Regexp *regexp.Regexp <HEX> ResyncInterval 90000000000 StatusReportingInterval 0 ConfigChangedRestartCallback func <HEX> FatalErrorRestartCallback func error <HEX> PostInSyncCallback func <HEX> HealthAggregator *health.HealthAggregator <HEX> RouteTableManager *idalloc.IndexAllocator <HEX> DebugSimulateDataplaneHangAfter 0 ExternalNodesCidrs string nil BPFEnabled false BPFDisableUnprivileged true BPFKubeProxyIptablesCleanupEnabled true BPFLogLevel off BPFExtToServiceConnmark 0 BPFDataIfacePattern *regexp.Regexp <HEX> XDPEnabled true XDPAllowGeneric false BPFConntrackTimeouts conntrack.Timeouts CreationGracePeriod 10000000000 TCPPreEstablished 20000000000 TCPEstablished 3600000000000 TCPFinsSeen 30000000000 TCPResetSeen 40000000000 UDPLastSeen 60000000000 GenericIPLastSeen 600000000000 ICMPLastSeen 5000000000 BPFCgroupV2 BPFConnTimeLBEnabled true BPFMapRepin true BPFNodePortDSREnabled false KubeProxyMinSyncPeriod 1000000000 KubeProxyEndpointSlicesEnabled false SidecarAccelerationEnabled false LookPathOverride func string string error nil KubeClientSet *kubernetes.Clientset <HEX> FeatureDetectOverrides map string string nil hostMTU 0 MTUIfacePattern *regexp.Regexp <HEX>
Reconciler error error unable to install crd unable to update version config map Operation cannot be fulfilled on configmaps palette-cert-manager-version-info the object has been modified; please apply your changes to the latest version and try again controller spectrocluster name ehs20testdeploy-5-10 namespace cluster-615c32c6cc841e9ea490508e
attempted to modprobe nf_conntrack_proto_sctp error exit status 1 output
Failed to get IPIP tunnel device assuming it isn t present error Link not found
Unable to remove old endpoints from kubernetes service StorageError key not found Code 1 Key /registry/masterleases/<IP> ResourceVersion 0 AdditionalErrorMsg
v1beta1.external.metrics.k8s.io failed with failing or missing response from https //<IP> 8443/apis/external.metrics.k8s.io/v1beta1 Get https //<IP> 8443/apis/external.metrics.k8s.io/v1beta1 net/http request canceled while waiting for connection Client.Timeout exceeded while awaiting headers
v1beta1.metrics.k8s.io failed with failing or missing response from https //<IP> 443/apis/metrics.k8s.io/v1beta1 Get https //<IP> 443/apis/metrics.k8s.io/v1beta1 context deadline exceeded
<*> failed with Operation cannot be fulfilled on apiservices.apiregistration.k8s.io <*> the object has been modified; please apply your changes to the latest version and try again
loading OpenAPI spec for v1beta1.external.metrics.k8s.io failed with OpenAPI spec does not exist
loading OpenAPI spec for v1beta1.metrics.k8s.io failed with failed to retrieve openAPI spec http error ResponseCode 503 Body error trying to reach service dial tcp <IP> 443 i/o timeout
error building openapi models for felixconfigurations.crd.projectcalico.org ERROR $root.definitions.org.projectcalico.crd.v1.FelixConfiguration.properties.spec.properties.kubeNodePortRanges.items.<array> has invalid property anyOf
ERROR $root.definitions.org.projectcalico.crd.v1.FelixConfiguration.properties.spec.properties.natPortRange has invalid property anyOf
Failed to add IPIP tunnel device error exit status 1
Failed configure IPIP tunnel device retrying... error exit status 1
error deleting general security groups for AWSManagedControlPlane error failed to describe cluster-owned security groups in vpc vpc-0d11162bb433f2453 RequestLimitExceeded Request limit exceeded. n tstatus code 503 request id 51a9abd1-fecd-467a-8b56-f78b8240224b name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Reconciler error error failed to describe cluster-owned security groups in vpc vpc-0d11162bb433f2453 RequestLimitExceeded Request limit exceeded. n tstatus code 503 request id 51a9abd1-fecd-467a-8b56-f78b8240224b controller awsmanagedcontrolplane name ehs20testdeploy-5-10-cp namespace cluster-615c32c6cc841e9ea490508e
Error proxying data from backend to client tls use of closed connection
failed to delete capi cluster error etcdserver request timed out spectrocluster Namespace cluster-615c32c6cc841e9ea490508e Name ehs20testdeploy-5-10
<DATE> <TIME> W | etcdserver/api/etcdhttp /health error; QGET failed etcdserver request timed out status code 503
Failed to access interface because it doesn t exist. error Link not found ifaceName califb68a382362 ifaceRegex ^cali.* ipVersion <HEX>
Failed to get interface; it s down/gone. error Link not found ifaceName califb68a382362 ifaceRegex ^cali.* ipVersion <HEX>
Failed to delete route error no such process ifaceName cali4d16b20af92 ifaceRegex ^cali.* ipVersion <HEX>
Interface was deleted during operation filtering error error netlink update operation failed ifaceName cali4d16b20af92 ifaceRegex ^cali.* ipVersion <HEX>
Error getting cluster information config ClusterInformation default error etcdserver leader changed
Failed to verify datastore error etcdserver leader changed
WARNING 2021/10/20 19 49 <*> grpc Server.processUnaryRPC failed to write status connection error desc transport is closing
<DATE> <TIME> W | etcdserver read-only range request key /registry/configmaps/cluster-61705ac0bbd3e5cf3bb47921/kubeadm-control-plane-manager-leader-election-capi with result error context canceled took too long 862.11444ms to execute
<DATE> <TIME> W | etcdserver read-only range request key <*> with result error context deadline exceeded took too long <*> to execute
Auditing failed of request encoding failed authentication.TokenRequest is not suitable for converting to v1 in scheme k8s.io/kubernetes/pkg/api/legacyscheme/scheme.go 30
Auditing failed of request encoding failed /v1 Kind DeleteOptions is unstructured and is not suitable for converting to <*>
Error getting cluster information config ClusterInformation default error etcdserver request timed out
Failed to verify datastore error etcdserver request timed out
error retrieving resource lock <*> etcdserver leader changed
Failed to reach apiserver error <nil>
Failed to update lock etcdserver request timed out
error retrieving resource lock kube-system/kube-scheduler etcdserver request timed out
error retrieving resource lock kube-system/kube-scheduler Get https //<IP> 6443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-scheduler?timeout 10s dial tcp <IP> 6443 connect connection refused
error retrieving resource lock kube-system/kube-controller-manager leases.coordination.k8s.io kube-controller-manager is forbidden User system kube-controller-manager cannot get resource leases in API group coordination.k8s.io in the namespace kube-system
<DATE> <TIME> E | etcdserver publish error etcdserver request timed out possibly due to connection lost
<DATE> <TIME> E | etcdserver publish error etcdserver request timed out
Failed to update statusUpdateNeeded field in actual state of world Failed to set statusUpdateNeeded to needed true because nodeName <*> does not exist
<DATE> <TIME> I | embed rejected connection from <IP> 56644 error EOF ServerName
<DATE> <TIME> I | embed rejected connection from <IP> 57038 error EOF ServerName etcd-ip-10-0-5-142.us-west-1.compute.internal
Failed to extract cronJobs list etcdserver request timed out
deletion of namespace cluster-609113a404b678bb4e8845c6 failed etcdserver leader changed
error deleting network for AWSManagedControlPlane error failed to query ec2 for VPCs RequestLimitExceeded Request limit exceeded. n tstatus code 503 request id 5a90ff8b-5132-42ba-8ec7-454de8cc74ad name ehs20deploy-cp namespace cluster-615d46c27cacf2b71bb3abec
Reconciler error error failed to query ec2 for VPCs RequestLimitExceeded Request limit exceeded. n tstatus code 503 request id 5a90ff8b-5132-42ba-8ec7-454de8cc74ad controller awsmanagedcontrolplane name ehs20deploy-cp namespace cluster-615d46c27cacf2b71bb3abec
apiserver received an error that is not an metav1.Status rpctypes.EtcdError code <HEX> desc etcdserver request timed out
<DATE> <TIME> W | etcdserver read-only range request key <*> with result error etcdserver leader changed took too long <*> to execute
<DATE> <TIME> W | etcdserver read-only range request key <*> range_end <*> count_only true with result error etcdserver leader changed took too long <*> to execute
<DATE> <TIME> W | etcdserver read-only range request key <*> range_end <*> with result error etcdserver leader changed took too long <*> to execute
<DATE> <TIME> W | etcdserver <*> range request key <*> with result error etcdserver request timed out took too long <*> to execute
<DATE> <TIME> W | etcdserver read-only range request key <*> range_end <*> count_only true with result error etcdserver request timed out took too long <*> to execute
<DATE> <TIME> W | etcdserver read-only range request key <*> range_end <*> limit <*> with result error etcdserver request timed out took too long <*> to execute
Error updating resource Key KubeControllersConfiguration default Name default Resource KubeControllersConfigurations Value &v3.KubeControllersConfiguration TypeMeta v1.TypeMeta Kind KubeControllersConfiguration APIVersion projectcalico.org/v3 ObjectMeta v1.ObjectMeta Name default GenerateName Namespace SelfLink UID 4dd88f1b-5220-4df9-ae1c-57f667cad299 ResourceVersion 372632857 Generation 0 CreationTimestamp v1.Time Time time.Time wall <HEX> ext 63743792777 loc *time.Location <HEX> DeletionTimestamp *v1.Time nil DeletionGracePeriodSeconds *int64 nil Labels map string string nil Annotations map string string nil OwnerReferences v1.OwnerReference nil Finalizers string nil ClusterName ManagedFields v1.ManagedFieldsEntry nil Spec v3.KubeControllersConfigurationSpec LogSeverityScreen Info HealthChecks Enabled EtcdV3CompactionPeriod *v1.Duration <HEX> PrometheusMetricsPort *int <HEX> Controllers v3.ControllersConfig Node *v3.NodeControllerConfig <HEX> Policy *v3.PolicyControllerConfig <HEX> WorkloadEndpoint *v3.WorkloadEndpointControllerConfig <HEX> ServiceAccount *v3.ServiceAccountControllerConfig <HEX> Namespace *v3.NamespaceControllerConfig <HEX> Status v3.KubeControllersConfigurationStatus RunningConfig v3.KubeControllersConfigurationSpec LogSeverityScreen Info HealthChecks Enabled EtcdV3CompactionPeriod *v1.Duration <HEX> PrometheusMetricsPort *int nil Controllers v3.ControllersConfig Node *v3.NodeControllerConfig <HEX> Policy *v3.PolicyControllerConfig nil WorkloadEndpoint *v3.WorkloadEndpointControllerConfig nil ServiceAccount *v3.ServiceAccountControllerConfig nil Namespace *v3.NamespaceControllerConfig nil EnvironmentVars map string string DATASTORE_TYPE kubernetes ENABLED_CONTROLLERS node error etcdserver leader changed
unable to perform status update on KubeControllersConfiguration default error etcdserver leader changed
Failed to reach apiserver error etcdserver leader changed
unable to get KubeControllersConfiguration default error etcdserver leader changed
Error getting cluster information config ClusterInformation default error Get https //<IP> 443/apis/crd.projectcalico.org/v1/clusterinformations/default context deadline exceeded
Failed to verify datastore error Get https //<IP> 443/apis/crd.projectcalico.org/v1/clusterinformations/default context deadline exceeded
<DATE> <TIME> W | etcdserver failed to revoke 30da7ab3e858fc3e etcdserver request timed out