Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploying rke cluster fails on provider version 1.3.2 #355

Open
Aransh opened this issue Jul 31, 2022 · 7 comments
Open

Deploying rke cluster fails on provider version 1.3.2 #355

Aransh opened this issue Jul 31, 2022 · 7 comments
Assignees
@a-blender
Labels
area/terraform bug Something isn't working team/area2
Milestone
v1.4.x

Comments

@Aransh
Copy link

Aransh commented Jul 31, 2022 •
edited
Loading

I am unable to deploy an rke cluster using version 1.3.2 of the provider (with either k8s v1.23.6-rancher1-1 or v1.21.7-rancher1-1).
Downgrading to provider version 1.3.1 works (but of course, means k8s 1.23 is not supported).

The main error I'm receiving is 'Failed running cluster err:[ "master-ip-address" not found]'
Again, the same exact configuration works just fine with provider version 1.3.1, so I doubt it's an infrastructure issue.
Also, I've been able to reproduce the issue on version 1.3.2 10 out of 10 times, while on 1.3.1 (and older versions) it worked every time for months.

Commenting the full error log (as it's longer than github's limit)
@Aransh
Copy link
Author

Aransh commented Jul 31, 2022 

Error:
============= RKE outputs ==============
time="2022-07-31T12:32:05+03:00" level=info msg="[rke_provider] rke cluster changed arguments: map[addons:true authentication:true bastion_host:true cloud_provider:true cluster_name:true ingress:true kubernetes_version:true network:true nodes:true restore:true services:true ssh_key_path:true]"
time="2022-07-31T12:32:05+03:00" level=info msg="Creating RKE cluster..."
time="2022-07-31T12:32:05+03:00" level=info msg="Initiating Kubernetes cluster"
time="2022-07-31T12:32:05+03:00" level=info msg="[dialer] Setup tunnel for host [10.145.2.233]"
time="2022-07-31T12:32:05+03:00" level=info msg="[dialer] Setup tunnel for host [10.145.6.29]"
time="2022-07-31T12:32:05+03:00" level=info msg="[dialer] Setup tunnel for host [10.145.4.102]"
time="2022-07-31T12:32:08+03:00" level=info msg="Checking if container [cluster-state-deployer] is running on host [10.145.2.233], try #1"
time="2022-07-31T12:32:08+03:00" level=info msg="Pulling image [rancher/rke-tools:v0.1.80] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:16+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:19+03:00" level=info msg="Starting container [cluster-state-deployer] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:19+03:00" level=info msg="[state] Successfully started [cluster-state-deployer] container on host [10.145.2.233]"
time="2022-07-31T12:32:20+03:00" level=info msg="Checking if container [cluster-state-deployer] is running on host [10.145.4.102], try #1"
time="2022-07-31T12:32:20+03:00" level=info msg="Pulling image [rancher/rke-tools:v0.1.80] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:28+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:30+03:00" level=info msg="Starting container [cluster-state-deployer] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:30+03:00" level=info msg="[state] Successfully started [cluster-state-deployer] container on host [10.145.4.102]"
time="2022-07-31T12:32:30+03:00" level=info msg="Checking if container [cluster-state-deployer] is running on host [10.145.6.29], try #1"
time="2022-07-31T12:32:31+03:00" level=info msg="Pulling image [rancher/rke-tools:v0.1.80] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:39+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:41+03:00" level=info msg="Starting container [cluster-state-deployer] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:41+03:00" level=info msg="[state] Successfully started [cluster-state-deployer] container on host [10.145.6.29]"
time="2022-07-31T12:32:41+03:00" level=info msg="[certificates] Generating CA kubernetes certificates"
time="2022-07-31T12:32:42+03:00" level=info msg="[certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates"
time="2022-07-31T12:32:42+03:00" level=info msg="[certificates] GenerateServingCertificate is disabled, checking if there are unused kubelet certificates"
time="2022-07-31T12:32:42+03:00" level=info msg="[certificates] Generating Kubernetes API server certificates"
time="2022-07-31T12:32:42+03:00" level=info msg="[certificates] Generating Service account token key"
time="2022-07-31T12:32:42+03:00" level=info msg="[certificates] Generating Kube Controller certificates"
time="2022-07-31T12:32:42+03:00" level=info msg="[certificates] Generating Kube Scheduler certificates"
time="2022-07-31T12:32:43+03:00" level=info msg="[certificates] Generating Kube Proxy certificates"
time="2022-07-31T12:32:43+03:00" level=info msg="[certificates] Generating Node certificate"
time="2022-07-31T12:32:43+03:00" level=info msg="[certificates] Generating admin certificates and kubeconfig"
time="2022-07-31T12:32:44+03:00" level=info msg="[certificates] Generating Kubernetes API server proxy client certificates"
time="2022-07-31T12:32:44+03:00" level=info msg="[certificates] Generating kube-etcd-10-145-2-233 certificate and key"
time="2022-07-31T12:32:44+03:00" level=info msg="[certificates] Generating kube-etcd-10-145-4-102 certificate and key"
time="2022-07-31T12:32:44+03:00" level=info msg="[certificates] Generating kube-etcd-10-145-6-29 certificate and key"
time="2022-07-31T12:32:44+03:00" level=info msg="Successfully Deployed state file at [/Users/aransh/Downloads/igentify/terraform-modules/regression_tests/kubernetes_cluster/terraform-provider-rke-tmp-280480591/cluster.rkestate]"
time="2022-07-31T12:32:44+03:00" level=info msg="Building Kubernetes cluster"
time="2022-07-31T12:32:44+03:00" level=info msg="[dialer] Setup tunnel for host [10.145.2.233]"
time="2022-07-31T12:32:44+03:00" level=info msg="[dialer] Setup tunnel for host [10.145.4.102]"
time="2022-07-31T12:32:44+03:00" level=info msg="[dialer] Setup tunnel for host [10.145.6.29]"
time="2022-07-31T12:32:47+03:00" level=info msg="[network] Deploying port listener containers"
time="2022-07-31T12:32:47+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:47+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:47+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:48+03:00" level=info msg="Starting container [rke-etcd-port-listener] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:48+03:00" level=info msg="Starting container [rke-etcd-port-listener] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:48+03:00" level=info msg="Starting container [rke-etcd-port-listener] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:48+03:00" level=info msg="[network] Successfully started [rke-etcd-port-listener] container on host [10.145.2.233]"
time="2022-07-31T12:32:48+03:00" level=info msg="[network] Successfully started [rke-etcd-port-listener] container on host [10.145.4.102]"
time="2022-07-31T12:32:49+03:00" level=info msg="[network] Successfully started [rke-etcd-port-listener] container on host [10.145.6.29]"
time="2022-07-31T12:32:49+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:49+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:49+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:49+03:00" level=info msg="Starting container [rke-cp-port-listener] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:49+03:00" level=info msg="Starting container [rke-cp-port-listener] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:49+03:00" level=info msg="Starting container [rke-cp-port-listener] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:50+03:00" level=info msg="[network] Successfully started [rke-cp-port-listener] container on host [10.145.2.233]"
time="2022-07-31T12:32:50+03:00" level=info msg="[network] Successfully started [rke-cp-port-listener] container on host [10.145.4.102]"
time="2022-07-31T12:32:50+03:00" level=info msg="[network] Successfully started [rke-cp-port-listener] container on host [10.145.6.29]"
time="2022-07-31T12:32:50+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:50+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:50+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:51+03:00" level=info msg="Starting container [rke-worker-port-listener] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:51+03:00" level=info msg="Starting container [rke-worker-port-listener] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:51+03:00" level=info msg="Starting container [rke-worker-port-listener] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Successfully started [rke-worker-port-listener] container on host [10.145.2.233]"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Successfully started [rke-worker-port-listener] container on host [10.145.6.29]"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Successfully started [rke-worker-port-listener] container on host [10.145.4.102]"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Port listener containers deployed successfully"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Running etcd <-> etcd port checks"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Checking if host [10.145.2.233] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [2379 2380], try #1"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Checking if host [10.145.4.102] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [2379 2380], try #1"
time="2022-07-31T12:32:51+03:00" level=info msg="[network] Checking if host [10.145.6.29] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [2379 2380], try #1"
time="2022-07-31T12:32:52+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:52+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:52+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:52+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:52+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:52+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:52+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.2.233]"
time="2022-07-31T12:32:52+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.4.102]"
time="2022-07-31T12:32:52+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.6.29]"
time="2022-07-31T12:32:53+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:53+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:53+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:53+03:00" level=info msg="[network] Running control plane -> etcd port checks"
time="2022-07-31T12:32:53+03:00" level=info msg="[network] Checking if host [10.145.2.233] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [2379], try #1"
time="2022-07-31T12:32:53+03:00" level=info msg="[network] Checking if host [10.145.6.29] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [2379], try #1"
time="2022-07-31T12:32:53+03:00" level=info msg="[network] Checking if host [10.145.4.102] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [2379], try #1"
time="2022-07-31T12:32:53+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:53+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:53+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:54+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.2.233]"
time="2022-07-31T12:32:54+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.4.102]"
time="2022-07-31T12:32:54+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.6.29]"
time="2022-07-31T12:32:54+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="[network] Running control plane -> worker port checks"
time="2022-07-31T12:32:54+03:00" level=info msg="[network] Checking if host [10.145.2.233] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [10250], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="[network] Checking if host [10.145.4.102] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [10250], try #1"
time="2022-07-31T12:32:54+03:00" level=info msg="[network] Checking if host [10.145.6.29] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [10250], try #1"
time="2022-07-31T12:32:55+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:55+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:55+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:55+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:55+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:55+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:56+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.2.233]"
time="2022-07-31T12:32:56+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.4.102]"
time="2022-07-31T12:32:56+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.6.29]"
time="2022-07-31T12:32:56+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:56+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:56+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:56+03:00" level=info msg="[network] Running workers -> control plane port checks"
time="2022-07-31T12:32:56+03:00" level=info msg="[network] Checking if host [10.145.4.102] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [6443], try #1"
time="2022-07-31T12:32:56+03:00" level=info msg="[network] Checking if host [10.145.2.233] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [6443], try #1"
time="2022-07-31T12:32:56+03:00" level=info msg="[network] Checking if host [10.145.6.29] can connect to host(s) [10.145.2.233 10.145.4.102 10.145.6.29] on port(s) [6443], try #1"
time="2022-07-31T12:32:56+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:32:56+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:32:56+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:32:57+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:57+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:57+03:00" level=info msg="Starting container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:57+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.2.233]"
time="2022-07-31T12:32:57+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.4.102]"
time="2022-07-31T12:32:57+03:00" level=info msg="[network] Successfully started [rke-port-checker] container on host [10.145.6.29]"
time="2022-07-31T12:32:57+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:57+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:57+03:00" level=info msg="Removing container [rke-port-checker] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:58+03:00" level=info msg="[network] Skipping kubeapi port check"
time="2022-07-31T12:32:58+03:00" level=info msg="[network] Removing port listener containers"
time="2022-07-31T12:32:58+03:00" level=info msg="Removing container [rke-etcd-port-listener] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:58+03:00" level=info msg="Removing container [rke-etcd-port-listener] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:58+03:00" level=info msg="Removing container [rke-etcd-port-listener] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:58+03:00" level=info msg="[remove/rke-etcd-port-listener] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:32:58+03:00" level=info msg="[remove/rke-etcd-port-listener] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:32:58+03:00" level=info msg="[remove/rke-etcd-port-listener] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:32:58+03:00" level=info msg="Removing container [rke-cp-port-listener] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:58+03:00" level=info msg="Removing container [rke-cp-port-listener] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:58+03:00" level=info msg="Removing container [rke-cp-port-listener] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:59+03:00" level=info msg="[remove/rke-cp-port-listener] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:32:59+03:00" level=info msg="[remove/rke-cp-port-listener] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:32:59+03:00" level=info msg="[remove/rke-cp-port-listener] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:32:59+03:00" level=info msg="Removing container [rke-worker-port-listener] on host [10.145.2.233], try #1"
time="2022-07-31T12:32:59+03:00" level=info msg="Removing container [rke-worker-port-listener] on host [10.145.4.102], try #1"
time="2022-07-31T12:32:59+03:00" level=info msg="Removing container [rke-worker-port-listener] on host [10.145.6.29], try #1"
time="2022-07-31T12:32:59+03:00" level=info msg="[remove/rke-worker-port-listener] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:32:59+03:00" level=info msg="[remove/rke-worker-port-listener] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:32:59+03:00" level=info msg="[remove/rke-worker-port-listener] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:32:59+03:00" level=info msg="[network] Port listener containers removed successfully"
time="2022-07-31T12:32:59+03:00" level=info msg="[certificates] Deploying kubernetes certificates to Cluster nodes"
time="2022-07-31T12:32:59+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.6.29], try #1"
time="2022-07-31T12:32:59+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.2.233], try #1"
time="2022-07-31T12:32:59+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.4.102], try #1"
time="2022-07-31T12:33:00+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:33:00+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:33:00+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:33:00+03:00" level=info msg="Starting container [cert-deployer] on host [10.145.2.233], try #1"
time="2022-07-31T12:33:00+03:00" level=info msg="Starting container [cert-deployer] on host [10.145.4.102], try #1"
time="2022-07-31T12:33:00+03:00" level=info msg="Starting container [cert-deployer] on host [10.145.6.29], try #1"
time="2022-07-31T12:33:00+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.2.233], try #1"
time="2022-07-31T12:33:01+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.4.102], try #1"
time="2022-07-31T12:33:01+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.6.29], try #1"
time="2022-07-31T12:33:06+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.2.233], try #1"
time="2022-07-31T12:33:06+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.4.102], try #1"
time="2022-07-31T12:33:06+03:00" level=info msg="Checking if container [cert-deployer] is running on host [10.145.6.29], try #1"

@Aransh
Copy link
Author

Aransh commented Jul 31, 2022 

time="2022-07-31T12:33:06+03:00" level=info msg="Removing container [cert-deployer] on host [10.145.2.233], try #1"
time="2022-07-31T12:33:06+03:00" level=info msg="Removing container [cert-deployer] on host [10.145.4.102], try #1"
time="2022-07-31T12:33:06+03:00" level=info msg="Removing container [cert-deployer] on host [10.145.6.29], try #1"
time="2022-07-31T12:33:06+03:00" level=info msg="[reconcile] Rebuilding and updating local kube config"
time="2022-07-31T12:33:06+03:00" level=info msg="Successfully Deployed local admin kubeconfig at [/Users/aransh/Downloads/igentify/terraform-modules/regression_tests/kubernetes_cluster/terraform-provider-rke-tmp-280480591/kube_config_cluster.yml]"
time="2022-07-31T12:33:07+03:00" level=warning msg="[reconcile] host [10.145.2.233] is a control plane node without reachable Kubernetes API endpoint in the cluster"
time="2022-07-31T12:33:07+03:00" level=info msg="Successfully Deployed local admin kubeconfig at [/Users/aransh/Downloads/igentify/terraform-modules/regression_tests/kubernetes_cluster/terraform-provider-rke-tmp-280480591/kube_config_cluster.yml]"
time="2022-07-31T12:33:08+03:00" level=warning msg="[reconcile] host [10.145.4.102] is a control plane node without reachable Kubernetes API endpoint in the cluster"
time="2022-07-31T12:33:08+03:00" level=info msg="Successfully Deployed local admin kubeconfig at [/Users/aransh/Downloads/igentify/terraform-modules/regression_tests/kubernetes_cluster/terraform-provider-rke-tmp-280480591/kube_config_cluster.yml]"
time="2022-07-31T12:33:10+03:00" level=warning msg="[reconcile] host [10.145.6.29] is a control plane node without reachable Kubernetes API endpoint in the cluster"
time="2022-07-31T12:33:10+03:00" level=warning msg="[reconcile] no control plane node with reachable Kubernetes API endpoint in the cluster found"
time="2022-07-31T12:33:10+03:00" level=info msg="[certificates] Successfully deployed kubernetes certificates to Cluster nodes"
time="2022-07-31T12:33:10+03:00" level=info msg="[file-deploy] Deploying file [/etc/kubernetes/cloud-config] to node [10.145.2.233]"
time="2022-07-31T12:33:10+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:33:10+03:00" level=info msg="Starting container [file-deployer] on host [10.145.2.233], try #1"
time="2022-07-31T12:33:11+03:00" level=info msg="Successfully started [file-deployer] container on host [10.145.2.233]"
time="2022-07-31T12:33:11+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.2.233]"
time="2022-07-31T12:33:11+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.2.233]"
time="2022-07-31T12:33:11+03:00" level=info msg="Removing container [file-deployer] on host [10.145.2.233], try #1"
time="2022-07-31T12:33:11+03:00" level=info msg="[remove/file-deployer] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:33:11+03:00" level=info msg="[file-deploy] Deploying file [/etc/kubernetes/cloud-config] to node [10.145.4.102]"
time="2022-07-31T12:33:12+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:33:12+03:00" level=info msg="Starting container [file-deployer] on host [10.145.4.102], try #1"
time="2022-07-31T12:33:13+03:00" level=info msg="Successfully started [file-deployer] container on host [10.145.4.102]"
time="2022-07-31T12:33:13+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.4.102]"
time="2022-07-31T12:33:13+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.4.102]"
time="2022-07-31T12:33:13+03:00" level=info msg="Removing container [file-deployer] on host [10.145.4.102], try #1"
time="2022-07-31T12:33:13+03:00" level=info msg="[remove/file-deployer] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:33:13+03:00" level=info msg="[file-deploy] Deploying file [/etc/kubernetes/cloud-config] to node [10.145.6.29]"
time="2022-07-31T12:33:14+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:33:14+03:00" level=info msg="Starting container [file-deployer] on host [10.145.6.29], try #1"
time="2022-07-31T12:33:15+03:00" level=info msg="Successfully started [file-deployer] container on host [10.145.6.29]"
time="2022-07-31T12:33:15+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.6.29]"
time="2022-07-31T12:33:15+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.6.29]"
time="2022-07-31T12:33:15+03:00" level=info msg="Removing container [file-deployer] on host [10.145.6.29], try #1"
time="2022-07-31T12:33:15+03:00" level=info msg="[remove/file-deployer] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:33:15+03:00" level=info msg="[/etc/kubernetes/cloud-config] Successfully deployed kubernetes cloud config to Cluster nodes"
time="2022-07-31T12:33:15+03:00" level=info msg="[file-deploy] Deploying file [/etc/kubernetes/audit-policy.yaml] to node [10.145.2.233]"
time="2022-07-31T12:33:16+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:33:16+03:00" level=info msg="Starting container [file-deployer] on host [10.145.2.233], try #1"
time="2022-07-31T12:33:17+03:00" level=info msg="Successfully started [file-deployer] container on host [10.145.2.233]"
time="2022-07-31T12:33:17+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.2.233]"
time="2022-07-31T12:33:17+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.2.233]"
time="2022-07-31T12:33:18+03:00" level=info msg="Removing container [file-deployer] on host [10.145.2.233], try #1"
time="2022-07-31T12:33:18+03:00" level=info msg="[remove/file-deployer] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:33:18+03:00" level=info msg="[file-deploy] Deploying file [/etc/kubernetes/audit-policy.yaml] to node [10.145.4.102]"
time="2022-07-31T12:33:18+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:33:18+03:00" level=info msg="Starting container [file-deployer] on host [10.145.4.102], try #1"
time="2022-07-31T12:33:19+03:00" level=info msg="Successfully started [file-deployer] container on host [10.145.4.102]"
time="2022-07-31T12:33:19+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.4.102]"
time="2022-07-31T12:33:19+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.4.102]"
time="2022-07-31T12:33:19+03:00" level=info msg="Removing container [file-deployer] on host [10.145.4.102], try #1"
time="2022-07-31T12:33:19+03:00" level=info msg="[remove/file-deployer] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:33:19+03:00" level=info msg="[file-deploy] Deploying file [/etc/kubernetes/audit-policy.yaml] to node [10.145.6.29]"
time="2022-07-31T12:33:20+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:33:20+03:00" level=info msg="Starting container [file-deployer] on host [10.145.6.29], try #1"
time="2022-07-31T12:33:21+03:00" level=info msg="Successfully started [file-deployer] container on host [10.145.6.29]"
time="2022-07-31T12:33:21+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.6.29]"
time="2022-07-31T12:33:21+03:00" level=info msg="Waiting for [file-deployer] container to exit on host [10.145.6.29]"
time="2022-07-31T12:33:22+03:00" level=info msg="Removing container [file-deployer] on host [10.145.6.29], try #1"
time="2022-07-31T12:33:22+03:00" level=info msg="[remove/file-deployer] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:33:22+03:00" level=info msg="[/etc/kubernetes/audit-policy.yaml] Successfully deployed audit policy file to Cluster control nodes"
time="2022-07-31T12:33:22+03:00" level=info msg="[reconcile] Reconciling cluster state"
time="2022-07-31T12:33:22+03:00" level=info msg="[reconcile] This is newly generated cluster"
time="2022-07-31T12:33:22+03:00" level=info msg="Pre-pulling kubernetes images"
time="2022-07-31T12:33:22+03:00" level=info msg="Pulling image [rancher/hyperkube:v1.23.6-rancher1] on host [10.145.2.233], try #1"
time="2022-07-31T12:33:22+03:00" level=info msg="Pulling image [rancher/hyperkube:v1.23.6-rancher1] on host [10.145.4.102], try #1"
time="2022-07-31T12:33:22+03:00" level=info msg="Pulling image [rancher/hyperkube:v1.23.6-rancher1] on host [10.145.6.29], try #1"
time="2022-07-31T12:33:57+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.4.102]"
time="2022-07-31T12:33:58+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.6.29]"
time="2022-07-31T12:33:59+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.2.233]"
time="2022-07-31T12:33:59+03:00" level=info msg="Kubernetes images pulled successfully"
time="2022-07-31T12:33:59+03:00" level=info msg="[etcd] Building up etcd plane.."
time="2022-07-31T12:33:59+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:34:04+03:00" level=info msg="Starting container [etcd-fix-perm] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:05+03:00" level=info msg="Successfully started [etcd-fix-perm] container on host [10.145.2.233]"
time="2022-07-31T12:34:05+03:00" level=info msg="Waiting for [etcd-fix-perm] container to exit on host [10.145.2.233]"
time="2022-07-31T12:34:05+03:00" level=info msg="Waiting for [etcd-fix-perm] container to exit on host [10.145.2.233]"
time="2022-07-31T12:34:05+03:00" level=info msg="Removing container [etcd-fix-perm] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:05+03:00" level=info msg="[remove/etcd-fix-perm] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:34:05+03:00" level=info msg="Pulling image [rancher/mirrored-coreos-etcd:v3.5.3] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:12+03:00" level=info msg="Image [rancher/mirrored-coreos-etcd:v3.5.3] exists on host [10.145.2.233]"
time="2022-07-31T12:34:13+03:00" level=info msg="Starting container [etcd] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:14+03:00" level=info msg="[etcd] Successfully started [etcd] container on host [10.145.2.233]"
time="2022-07-31T12:34:14+03:00" level=info msg="[etcd] Snapshots configured to S3 compatible backend at [s3.amazonaws.com] to bucket [igentify-test-kubernetes-etcd-backup] using accesskey [AKIAXRCQ6HXIS4PBSLQY] and using region [eu-west-1]"
time="2022-07-31T12:34:14+03:00" level=info msg="[etcd] Running rolling snapshot container [etcd-snapshot-once] on host [10.145.2.233]"
time="2022-07-31T12:34:14+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:34:14+03:00" level=info msg="Starting container [etcd-rolling-snapshots] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:15+03:00" level=info msg="[etcd] Successfully started [etcd-rolling-snapshots] container on host [10.145.2.233]"
time="2022-07-31T12:34:20+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:34:21+03:00" level=info msg="Starting container [rke-bundle-cert] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:21+03:00" level=info msg="[certificates] Successfully started [rke-bundle-cert] container on host [10.145.2.233]"
time="2022-07-31T12:34:21+03:00" level=info msg="Waiting for [rke-bundle-cert] container to exit on host [10.145.2.233]"
time="2022-07-31T12:34:21+03:00" level=info msg="[certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [10.145.2.233]"
time="2022-07-31T12:34:21+03:00" level=info msg="Removing container [rke-bundle-cert] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:22+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:34:22+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:23+03:00" level=info msg="[etcd] Successfully started [rke-log-linker] container on host [10.145.2.233]"
time="2022-07-31T12:34:23+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:23+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:34:23+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:34:24+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:24+03:00" level=info msg="[etcd] Successfully started [rke-log-linker] container on host [10.145.2.233]"
time="2022-07-31T12:34:24+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:34:25+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:34:25+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:34:28+03:00" level=info msg="Starting container [etcd-fix-perm] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:28+03:00" level=info msg="Successfully started [etcd-fix-perm] container on host [10.145.4.102]"
time="2022-07-31T12:34:28+03:00" level=info msg="Waiting for [etcd-fix-perm] container to exit on host [10.145.4.102]"
time="2022-07-31T12:34:28+03:00" level=info msg="Waiting for [etcd-fix-perm] container to exit on host [10.145.4.102]"
time="2022-07-31T12:34:29+03:00" level=info msg="Removing container [etcd-fix-perm] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:29+03:00" level=info msg="[remove/etcd-fix-perm] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:34:29+03:00" level=info msg="Pulling image [rancher/mirrored-coreos-etcd:v3.5.3] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:35+03:00" level=info msg="Image [rancher/mirrored-coreos-etcd:v3.5.3] exists on host [10.145.4.102]"
time="2022-07-31T12:34:36+03:00" level=info msg="Starting container [etcd] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:37+03:00" level=info msg="[etcd] Successfully started [etcd] container on host [10.145.4.102]"
time="2022-07-31T12:34:37+03:00" level=info msg="[etcd] Snapshots configured to S3 compatible backend at [s3.amazonaws.com] to bucket [igentify-test-kubernetes-etcd-backup] using accesskey [AKIAXRCQ6HXIS4PBSLQY] and using region [eu-west-1]"
time="2022-07-31T12:34:37+03:00" level=info msg="[etcd] Running rolling snapshot container [etcd-snapshot-once] on host [10.145.4.102]"
time="2022-07-31T12:34:37+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:34:37+03:00" level=info msg="Starting container [etcd-rolling-snapshots] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:38+03:00" level=info msg="[etcd] Successfully started [etcd-rolling-snapshots] container on host [10.145.4.102]"
time="2022-07-31T12:34:43+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:34:44+03:00" level=info msg="Starting container [rke-bundle-cert] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:44+03:00" level=info msg="[certificates] Successfully started [rke-bundle-cert] container on host [10.145.4.102]"
time="2022-07-31T12:34:44+03:00" level=info msg="Waiting for [rke-bundle-cert] container to exit on host [10.145.4.102]"
time="2022-07-31T12:34:44+03:00" level=info msg="[certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [10.145.4.102]"
time="2022-07-31T12:34:44+03:00" level=info msg="Removing container [rke-bundle-cert] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:45+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:34:45+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:46+03:00" level=info msg="[etcd] Successfully started [rke-log-linker] container on host [10.145.4.102]"
time="2022-07-31T12:34:46+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:46+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:34:47+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:34:47+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:48+03:00" level=info msg="[etcd] Successfully started [rke-log-linker] container on host [10.145.4.102]"
time="2022-07-31T12:34:48+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:34:48+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:34:51+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:34:51+03:00" level=info msg="Starting container [etcd-fix-perm] on host [10.145.6.29], try #1"
time="2022-07-31T12:34:52+03:00" level=info msg="Successfully started [etcd-fix-perm] container on host [10.145.6.29]"
time="2022-07-31T12:34:52+03:00" level=info msg="Waiting for [etcd-fix-perm] container to exit on host [10.145.6.29]"
time="2022-07-31T12:34:52+03:00" level=info msg="Waiting for [etcd-fix-perm] container to exit on host [10.145.6.29]"
time="2022-07-31T12:34:52+03:00" level=info msg="Removing container [etcd-fix-perm] on host [10.145.6.29], try #1"
time="2022-07-31T12:34:52+03:00" level=info msg="[remove/etcd-fix-perm] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:34:52+03:00" level=info msg="Pulling image [rancher/mirrored-coreos-etcd:v3.5.3] on host [10.145.6.29], try #1"
time="2022-07-31T12:34:59+03:00" level=info msg="Image [rancher/mirrored-coreos-etcd:v3.5.3] exists on host [10.145.6.29]"
time="2022-07-31T12:35:00+03:00" level=info msg="Starting container [etcd] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:00+03:00" level=info msg="[etcd] Successfully started [etcd] container on host [10.145.6.29]"
time="2022-07-31T12:35:00+03:00" level=info msg="[etcd] Snapshots configured to S3 compatible backend at [s3.amazonaws.com] to bucket [igentify-test-kubernetes-etcd-backup] using accesskey [AKIAXRCQ6HXIS4PBSLQY] and using region [eu-west-1]"
time="2022-07-31T12:35:00+03:00" level=info msg="[etcd] Running rolling snapshot container [etcd-snapshot-once] on host [10.145.6.29]"
time="2022-07-31T12:35:01+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:01+03:00" level=info msg="Starting container [etcd-rolling-snapshots] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:02+03:00" level=info msg="[etcd] Successfully started [etcd-rolling-snapshots] container on host [10.145.6.29]"
time="2022-07-31T12:35:07+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:08+03:00" level=info msg="Starting container [rke-bundle-cert] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:08+03:00" level=info msg="[certificates] Successfully started [rke-bundle-cert] container on host [10.145.6.29]"
time="2022-07-31T12:35:08+03:00" level=info msg="Waiting for [rke-bundle-cert] container to exit on host [10.145.6.29]"
time="2022-07-31T12:35:08+03:00" level=info msg="[certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [10.145.6.29]"
time="2022-07-31T12:35:08+03:00" level=info msg="Removing container [rke-bundle-cert] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:09+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:09+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:10+03:00" level=info msg="[etcd] Successfully started [rke-log-linker] container on host [10.145.6.29]"
time="2022-07-31T12:35:10+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:10+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:35:11+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:11+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:11+03:00" level=info msg="[etcd] Successfully started [rke-log-linker] container on host [10.145.6.29]"
time="2022-07-31T12:35:12+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:12+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:35:12+03:00" level=info msg="[etcd] Successfully started etcd plane.. Checking etcd cluster health"
time="2022-07-31T12:35:14+03:00" level=info msg="[etcd] etcd host [10.145.2.233] reported healthy=true"
time="2022-07-31T12:35:14+03:00" level=info msg="[controlplane] Building up Controller Plane.."
time="2022-07-31T12:35:14+03:00" level=info msg="Checking if container [service-sidekick] is running on host [10.145.2.233], try #1"
time="2022-07-31T12:35:14+03:00" level=info msg="Checking if container [service-sidekick] is running on host [10.145.4.102], try #1"
time="2022-07-31T12:35:14+03:00" level=info msg="Checking if container [service-sidekick] is running on host [10.145.6.29], try #1"
time="2022-07-31T12:35:15+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:35:15+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:35:15+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:15+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.2.233]"
time="2022-07-31T12:35:15+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.6.29]"
time="2022-07-31T12:35:15+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.4.102]"
time="2022-07-31T12:35:15+03:00" level=info msg="Starting container [kube-apiserver] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:16+03:00" level=info msg="Starting container [kube-apiserver] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:16+03:00" level=info msg="Starting container [kube-apiserver] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:16+03:00" level=info msg="[controlplane] Successfully started [kube-apiserver] container on host [10.145.2.233]"
time="2022-07-31T12:35:16+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-apiserver] on host [10.145.2.233]"
time="2022-07-31T12:35:16+03:00" level=info msg="[controlplane] Successfully started [kube-apiserver] container on host [10.145.4.102]"
time="2022-07-31T12:35:16+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-apiserver] on host [10.145.4.102]"
time="2022-07-31T12:35:16+03:00" level=info msg="[controlplane] Successfully started [kube-apiserver] container on host [10.145.6.29]"
time="2022-07-31T12:35:16+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-apiserver] on host [10.145.6.29]"
time="2022-07-31T12:35:24+03:00" level=info msg="[healthcheck] service [kube-apiserver] on host [10.145.2.233] is healthy"
time="2022-07-31T12:35:24+03:00" level=info msg="[healthcheck] service [kube-apiserver] on host [10.145.4.102] is healthy"
time="2022-07-31T12:35:24+03:00" level=info msg="[healthcheck] service [kube-apiserver] on host [10.145.6.29] is healthy"
time="2022-07-31T12:35:25+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:35:25+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:35:25+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:25+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:25+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:25+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:26+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.2.233]"
time="2022-07-31T12:35:26+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.4.102]"
time="2022-07-31T12:35:26+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.6.29]"
time="2022-07-31T12:35:26+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:26+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:26+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:35:26+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:26+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:35:26+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:35:26+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.2.233]"
time="2022-07-31T12:35:26+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.4.102]"
time="2022-07-31T12:35:26+03:00" level=info msg="Starting container [kube-controller-manager] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:26+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.6.29]"
time="2022-07-31T12:35:26+03:00" level=info msg="Starting container [kube-controller-manager] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:27+03:00" level=info msg="Starting container [kube-controller-manager] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:27+03:00" level=info msg="[controlplane] Successfully started [kube-controller-manager] container on host [10.145.2.233]"
time="2022-07-31T12:35:27+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-controller-manager] on host [10.145.2.233]"
time="2022-07-31T12:35:27+03:00" level=info msg="[controlplane] Successfully started [kube-controller-manager] container on host [10.145.4.102]"
time="2022-07-31T12:35:27+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-controller-manager] on host [10.145.4.102]"
time="2022-07-31T12:35:27+03:00" level=info msg="[controlplane] Successfully started [kube-controller-manager] container on host [10.145.6.29]"
time="2022-07-31T12:35:27+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-controller-manager] on host [10.145.6.29]"
time="2022-07-31T12:35:29+03:00" level=info msg="[healthcheck] service [kube-controller-manager] on host [10.145.2.233] is healthy"
time="2022-07-31T12:35:29+03:00" level=info msg="[healthcheck] service [kube-controller-manager] on host [10.145.6.29] is healthy"
time="2022-07-31T12:35:29+03:00" level=info msg="[healthcheck] service [kube-controller-manager] on host [10.145.4.102] is healthy"
time="2022-07-31T12:35:30+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:35:30+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:30+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:30+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:35:30+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:30+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:30+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.2.233]"
time="2022-07-31T12:35:31+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:31+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:35:31+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.6.29]"
time="2022-07-31T12:35:31+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.4.102]"
time="2022-07-31T12:35:31+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.2.233]"
time="2022-07-31T12:35:31+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:31+03:00" level=info msg="Starting container [kube-scheduler] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:31+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:35:31+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:31+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:35:31+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.6.29]"
time="2022-07-31T12:35:32+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.4.102]"
time="2022-07-31T12:35:32+03:00" level=info msg="[controlplane] Successfully started [kube-scheduler] container on host [10.145.2.233]"
time="2022-07-31T12:35:32+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-scheduler] on host [10.145.2.233]"
time="2022-07-31T12:35:32+03:00" level=info msg="Starting container [kube-scheduler] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:32+03:00" level=info msg="Starting container [kube-scheduler] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:32+03:00" level=info msg="[controlplane] Successfully started [kube-scheduler] container on host [10.145.4.102]"
time="2022-07-31T12:35:32+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-scheduler] on host [10.145.4.102]"
time="2022-07-31T12:35:32+03:00" level=info msg="[controlplane] Successfully started [kube-scheduler] container on host [10.145.6.29]"
time="2022-07-31T12:35:32+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-scheduler] on host [10.145.6.29]"
time="2022-07-31T12:35:34+03:00" level=info msg="[healthcheck] service [kube-scheduler] on host [10.145.2.233] is healthy"
time="2022-07-31T12:35:35+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:35:35+03:00" level=info msg="[healthcheck] service [kube-scheduler] on host [10.145.4.102] is healthy"
time="2022-07-31T12:35:35+03:00" level=info msg="[healthcheck] service [kube-scheduler] on host [10.145.6.29] is healthy"
time="2022-07-31T12:35:35+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:35+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:35:35+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:36+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:36+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:36+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.2.233]"
time="2022-07-31T12:35:36+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:36+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:35:36+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.4.102]"
time="2022-07-31T12:35:36+03:00" level=info msg="[controlplane] Successfully started [rke-log-linker] container on host [10.145.6.29]"
time="2022-07-31T12:35:36+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:36+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:35:36+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:37+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:35:37+03:00" level=info msg="[controlplane] Successfully started Controller Plane.."
time="2022-07-31T12:35:37+03:00" level=info msg="[authz] Creating rke-job-deployer ServiceAccount"
time="2022-07-31T12:35:38+03:00" level=info msg="[authz] rke-job-deployer ServiceAccount created successfully"
time="2022-07-31T12:35:38+03:00" level=info msg="[authz] Creating system:node ClusterRoleBinding"
time="2022-07-31T12:35:39+03:00" level=info msg="[authz] system:node ClusterRoleBinding created successfully"
time="2022-07-31T12:35:39+03:00" level=info msg="[authz] Creating kube-apiserver proxy ClusterRole and ClusterRoleBinding"
time="2022-07-31T12:35:39+03:00" level=info msg="[authz] kube-apiserver proxy ClusterRole and ClusterRoleBinding created successfully"
time="2022-07-31T12:35:39+03:00" level=info msg="Successfully Deployed state file at [/Users/aransh/Downloads/igentify/terraform-modules/regression_tests/kubernetes_cluster/terraform-provider-rke-tmp-280480591/cluster.rkestate]"
time="2022-07-31T12:35:39+03:00" level=info msg="[state] Saving full cluster state to Kubernetes"
time="2022-07-31T12:35:41+03:00" level=info msg="[state] Successfully Saved full cluster state to Kubernetes ConfigMap: full-cluster-state"
time="2022-07-31T12:35:41+03:00" level=info msg="[worker] Building up Worker Plane.."
time="2022-07-31T12:35:41+03:00" level=info msg="Checking if container [service-sidekick] is running on host [10.145.6.29], try #1"
time="2022-07-31T12:35:41+03:00" level=info msg="Checking if container [service-sidekick] is running on host [10.145.4.102], try #1"
time="2022-07-31T12:35:41+03:00" level=info msg="Checking if container [service-sidekick] is running on host [10.145.2.233], try #1"
time="2022-07-31T12:35:41+03:00" level=info msg="[sidekick] Sidekick container already created on host [10.145.2.233]"
time="2022-07-31T12:35:41+03:00" level=info msg="[sidekick] Sidekick container already created on host [10.145.4.102]"
time="2022-07-31T12:35:41+03:00" level=info msg="[sidekick] Sidekick container already created on host [10.145.6.29]"
time="2022-07-31T12:35:41+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.2.233]"
time="2022-07-31T12:35:41+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.4.102]"
time="2022-07-31T12:35:41+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.6.29]"
time="2022-07-31T12:35:41+03:00" level=info msg="Starting container [kubelet] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:41+03:00" level=info msg="Starting container [kubelet] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:42+03:00" level=info msg="Starting container [kubelet] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:42+03:00" level=info msg="[worker] Successfully started [kubelet] container on host [10.145.2.233]"
time="2022-07-31T12:35:42+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kubelet] on host [10.145.2.233]"
time="2022-07-31T12:35:42+03:00" level=info msg="[worker] Successfully started [kubelet] container on host [10.145.4.102]"
time="2022-07-31T12:35:42+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kubelet] on host [10.145.4.102]"
time="2022-07-31T12:35:42+03:00" level=info msg="[worker] Successfully started [kubelet] container on host [10.145.6.29]"
time="2022-07-31T12:35:42+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kubelet] on host [10.145.6.29]"
time="2022-07-31T12:35:44+03:00" level=info msg="[healthcheck] service [kubelet] on host [10.145.4.102] is healthy"
time="2022-07-31T12:35:44+03:00" level=info msg="[healthcheck] service [kubelet] on host [10.145.2.233] is healthy"
time="2022-07-31T12:35:45+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:35:45+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:35:45+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:45+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:45+03:00" level=info msg="[worker] Successfully started [rke-log-linker] container on host [10.145.4.102]"
time="2022-07-31T12:35:45+03:00" level=info msg="[worker] Successfully started [rke-log-linker] container on host [10.145.2.233]"
time="2022-07-31T12:35:46+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:46+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:46+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:35:46+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:35:46+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.4.102]"
time="2022-07-31T12:35:46+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.2.233]"
time="2022-07-31T12:35:46+03:00" level=info msg="Starting container [kube-proxy] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:46+03:00" level=info msg="Starting container [kube-proxy] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:46+03:00" level=info msg="[worker] Successfully started [kube-proxy] container on host [10.145.4.102]"
time="2022-07-31T12:35:46+03:00" level=info msg="[worker] Successfully started [kube-proxy] container on host [10.145.2.233]"
time="2022-07-31T12:35:46+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-proxy] on host [10.145.4.102]"
time="2022-07-31T12:35:46+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-proxy] on host [10.145.2.233]"
time="2022-07-31T12:35:49+03:00" level=info msg="[healthcheck] service [kube-proxy] on host [10.145.4.102] is healthy"
time="2022-07-31T12:35:49+03:00" level=info msg="[healthcheck] service [kube-proxy] on host [10.145.2.233] is healthy"
time="2022-07-31T12:35:49+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:35:49+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:35:50+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:50+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:50+03:00" level=info msg="[worker] Successfully started [rke-log-linker] container on host [10.145.4.102]"
time="2022-07-31T12:35:51+03:00" level=info msg="[worker] Successfully started [rke-log-linker] container on host [10.145.2.233]"
time="2022-07-31T12:35:51+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:51+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:51+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:35:51+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:35:51+03:00" level=info msg="[healthcheck] service [kubelet] on host [10.145.6.29] is healthy"
time="2022-07-31T12:35:52+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:52+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:53+03:00" level=info msg="[worker] Successfully started [rke-log-linker] container on host [10.145.6.29]"
time="2022-07-31T12:35:53+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:53+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:35:53+03:00" level=info msg="Image [rancher/hyperkube:v1.23.6-rancher1] exists on host [10.145.6.29]"
time="2022-07-31T12:35:54+03:00" level=info msg="Starting container [kube-proxy] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:54+03:00" level=info msg="[worker] Successfully started [kube-proxy] container on host [10.145.6.29]"
time="2022-07-31T12:35:54+03:00" level=info msg="[healthcheck] Start Healthcheck on service [kube-proxy] on host [10.145.6.29]"
time="2022-07-31T12:35:56+03:00" level=info msg="[healthcheck] service [kube-proxy] on host [10.145.6.29] is healthy"
time="2022-07-31T12:35:57+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:57+03:00" level=info msg="Starting container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:58+03:00" level=info msg="[worker] Successfully started [rke-log-linker] container on host [10.145.6.29]"
time="2022-07-31T12:35:58+03:00" level=info msg="Removing container [rke-log-linker] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:58+03:00" level=info msg="[remove/rke-log-linker] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:35:58+03:00" level=info msg="[worker] Successfully started Worker Plane.."
time="2022-07-31T12:35:58+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.4.102]"
time="2022-07-31T12:35:58+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.2.233]"
time="2022-07-31T12:35:58+03:00" level=info msg="Image [rancher/rke-tools:v0.1.80] exists on host [10.145.6.29]"
time="2022-07-31T12:35:59+03:00" level=info msg="Starting container [rke-log-cleaner] on host [10.145.2.233], try #1"
time="2022-07-31T12:35:59+03:00" level=info msg="Starting container [rke-log-cleaner] on host [10.145.4.102], try #1"
time="2022-07-31T12:35:59+03:00" level=info msg="Starting container [rke-log-cleaner] on host [10.145.6.29], try #1"
time="2022-07-31T12:35:59+03:00" level=info msg="[cleanup] Successfully started [rke-log-cleaner] container on host [10.145.2.233]"
time="2022-07-31T12:35:59+03:00" level=info msg="[cleanup] Successfully started [rke-log-cleaner] container on host [10.145.4.102]"
time="2022-07-31T12:36:00+03:00" level=info msg="[cleanup] Successfully started [rke-log-cleaner] container on host [10.145.6.29]"
time="2022-07-31T12:36:00+03:00" level=info msg="Removing container [rke-log-cleaner] on host [10.145.4.102], try #1"
time="2022-07-31T12:36:00+03:00" level=info msg="Removing container [rke-log-cleaner] on host [10.145.2.233], try #1"
time="2022-07-31T12:36:00+03:00" level=info msg="[remove/rke-log-cleaner] Successfully removed container on host [10.145.2.233]"
time="2022-07-31T12:36:00+03:00" level=info msg="[remove/rke-log-cleaner] Successfully removed container on host [10.145.4.102]"
time="2022-07-31T12:36:00+03:00" level=info msg="Removing container [rke-log-cleaner] on host [10.145.6.29], try #1"
time="2022-07-31T12:36:00+03:00" level=info msg="[remove/rke-log-cleaner] Successfully removed container on host [10.145.6.29]"
time="2022-07-31T12:36:00+03:00" level=info msg="[sync] Syncing nodes Labels and Taints"

Failed running cluster err:[ "10.145.4.102" not found]
========================================


  on ../../kubernetes_cluster/rke_cluster.tf line 26, in resource "rke_cluster" "cluster":
  26: resource "rke_cluster" "cluster" {

@Aransh Aransh changed the title deploying rke cluster on provider version 1.3.2 deploying rke cluster fails on provider version 1.3.2 Jul 31, 2022
@Aransh
Copy link
Author

Aransh commented Aug 31, 2022

Update - this is still happening with version 1.3.3

@richiesebastian
Copy link

facing same issue - do we have a solution yet ?

@johnhpatton
Copy link

johnhpatton commented Jan 9, 2023 •
edited
Loading

Same issue here on provider version 1.3.4 using AWS EC2s.

@a-blender a-blender changed the title deploying rke cluster fails on provider version 1.3.2 Deploying rke cluster fails on provider version 1.3.2 Oct 30, 2023
@a-blender
Copy link
Contributor

a-blender commented Oct 30, 2023 •
edited
Loading

@Aransh Are you still seeing this issue? Can you please try on the latest version v1.3.5?

@a-blender a-blender added this to the v1.4.4 milestone Oct 30, 2023
@a-blender a-blender self-assigned this Oct 30, 2023
@Aransh
Copy link
Author

Aransh commented Oct 31, 2023

@a-blender posted this over a year ago 😅 No longer working at where this was a problem, maybe @johnhpatton or @richiesebastian can assist?

@Jono-SUSE-Rancher Jono-SUSE-Rancher modified the milestones: v1.4.4, v1.4.5 Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@a-blender a-blender

Labels
area/terraform bug Something isn't working team/area2
Projects
None yet
Milestone
v1.4.x
Development

No branches or pull requests
5 participants
@johnhpatton @Aransh @a-blender @Jono-SUSE-Rancher @richiesebastian