User cannot get configmaps in the namespace #27

sathyu · 2019-01-29T21:28:02Z

Hello,
I am deploying this in k8 and getting following errors, any idea what that I am missing?
I am deploying this as non-admin user.

$ kubectl logs stolon-sentinel-7754964b89-8vmv4
2019-01-29T21:22:26.067Z INFO cmd/sentinel.go:1962 sentinel uid {"uid": "dfa105e8"}
2019-01-29T21:22:26.071Z INFO cmd/sentinel.go:80 Trying to acquire sentinels leadership
ERROR: logging before flag.Parse: I0129 21:22:26.071509 1 leaderelection.go:174] attempting to acquire leader lease...
ERROR: logging before flag.Parse: E0129 21:22:26.146967 1 leaderelection.go:224] error retrieving resource lock k8poc-sathya/stolon-cluster-kube-stolon: configmaps "stolon-cluster-kube-stolon" is forbidden: User "system:serviceaccount:k8poc-sathya:default" cannot get configmaps in the namespace "k8poc-sathya"

lwolf · 2019-01-29T21:47:49Z

Hi,
as log record says

User "system:serviceaccount:k8poc-sathya:default" cannot get configmaps in the namespace "k8poc-sathya"

the user does not have permissions to get configmaps. You need to check ServiceAccount/Role/RoleBinding to make sure that the user has required permissions.

sathyu · 2019-02-01T17:15:48Z

Hello ,
As understanding, I can do all admin works within my name space. and I was told to user my service account which I am doing. Still getting error.
So, question 1. we I need access to kube-system namespace, if so , is just "Read" sufficient.
2. if I don't get access to kube-system what is the way to implement this ?
3. Are you available for 1:1 talk , I am ready to compensate your time (serious).

$ Error from server (Forbidden): deployments.extensions is forbidden: User "system:serviceaccount:k8poc-sathya:k8-poc-sathya" cannot list deployments.extensions in the namespace "kube-system"

lwolf · 2019-02-03T19:49:57Z

It shouldn't require access to the kube-system namespace at all.
Could you please provide more information about your setup, will see what I can do?

what k8s version do you use?
what stolon version do you use?
what namespace did you install the chart?
could you post content of your values.yaml stripping out (if any) sensitive information

sathyu · 2019-02-04T17:26:34Z

Step1:
$ kubectl version

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.5", GitCommit:"32ac1c9073b132b8ba18aa830f46b77dcceb0723", GitTreeState:"clean", BuildDate:"2018-06-21T11:46:00Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0", GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", GitTreeState:"clean", BuildDate:"2018-03-26T16:44:10Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

Step 2:
imageTag: "v0.12.0-pg10"
Namespace = k8poc-sathya
Step 3:
$ helm install --name k8poc-sathya -f values.yaml .
Error: pods is forbidden: User "system:serviceaccount:k8poc-sathya:k8-poc-sathya" cannot list pods in the namespace "kube-system"

step 4: values.yaml

Hello.
Below is the values.yaml entries:
$ cat values.yaml

# Default values for Stolon Helm Chart.
# This is a YAML-formatted file.
## Declare variables to be passed into your templates.

## Override the name of the Chart.
##
# nameOverride:

## Stolon image.
##

image: "sorintlab/stolon"

## Stolon image version.
## ref: https://hub.docker.com/r/sorintlab/stolon/tags/
##
imageTag: "v0.12.0-pg10"

## Specify a imagePullPolicy: 'Always' if imageTag is 'latest', else set to 'IfNotPresent'.
## ref: https://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
# imagePullPolicy:

## Configuration values for Stolon.

# Set custom stolon cluster name
clusterName: "kube-stolon"
debug: false

## log slow queries
# disabled by default
slow_queries:
  enabled: false
  min_duration: 300

ports:
  internalPort: 5432
  externalPort: 5432

store:
  ##  Backend could be one of the following:
  ## - etcdv2
  ## - etcdv3
  ## - consul (should work, but not tested yet)
  ## - kubernetes (should work, but not tested yet)
  backend: kubernetes
  ## store endpoints MUST be set for etcd/consul backends
  #  endpoints: "http://etcd-etcd-0.etcd-etcd:2379,http://etcd-etcd-1.etcd-etcd:2379,http://etcd-etcd-2.etcd-etcd:2379"

pgReplUsername: "repluser"
## set password for the repluser
## default is 40 random chars
pgReplPassword: "replPassword"

pgSuperuserName: "stolon"

## set password for the superuser
## default is 40 random chars
pgSuperuserPassword: "stolon123"

sentinel:
  replicas: 3

  ## Configure resource requests and limits.
  ## ref: https://kubernetes.io/docs/user-guide/compute-resources/
  ##

  resources:
    requests:
      cpu: "100m"
      memory: "512Mi"

  ## Configure nodeSelector, tolerations and affinity.
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node
  ##

  affinity: {}
  nodeSelector: {}
  tolerations: []

proxy:
  replicas: 2
  ## Set serviceType to nodePort if needed
  ## proxy is used to route RW requests to the master
  # serviceType: NodePort

  ## Configure resource requests and limits.
  ## ref: https://kubernetes.io/docs/user-guide/compute-resources/
  ##

  resources:
    requests:
      cpu: "100m"
      memory: "512Mi"

  ## Configure nodeSelector, tolerations and affinity.
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node
  ##

  affinity: {}
  nodeSelector: {}
  tolerations: []

keeper:
  replicas: 2
  ## Set serviceType to nodePort if needed
  ## keeper service is used to route RO requests to all nodes
  # serviceType: NodePort

  ## configure ssl for client access
  # create certificates according to these instructions: https://www.postgresql.org/docs/9.6/static/ssl-tcp.html
  # to enable encrypted traffic, servert.crt and server.key are required, by that name.
  # the use of ** Client Certificates ** is not supported
  client_ssl:
    enabled: false
    certs_secret_name: pg-cert-secret

  ## Configure resource requests and limits.
  ## ref: https://kubernetes.io/docs/user-guide/compute-resources/
  ##

  resources:
    requests:
      cpu: "100m"
      memory: "512Mi"

  ## Configure nodeSelector, tolerations and affinity.
  ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node
  ##

  affinity: {}
  nodeSelector: {}
  tolerations: []
## Persistent Volume Storage configuration.
## ref: https://kubernetes.io/docs/user-guide/persistent-volumes
##

persistence:
  ## Enable persistence using Persistent Volume Claims.
  ##
  enabled: false

  ## Persistent Volume Access Mode.
  ##
  accessMode: ReadWriteOnce

  ## Persistant Volume Storage Class Name
  storageClassName: standard

  ## Persistent Volume Storage Size.
  ##
  size: 25Gi

rbac:
  # Specifies whether RBAC resources should be created
  create: true

serviceAccount:
  # Specifies whether a ServiceAccount should be created
  create: true
  # The name of the ServiceAccount to use.
  # If not set and create is true, a name is generated using the fullname template
  name: k8-poc-sathya

lwolf · 2019-02-05T19:40:29Z

I've just deployed this chart with the values.yaml file you've pasted. I don't have k8s 1.10 so I tested on my 1.12.3. Everything works fine.

$ kubectl get pods
k8poc-sathya-stolon-keeper-0                                      1/1       Running     0          1m
k8poc-sathya-stolon-keeper-1                                      1/1       Running     0          46s
k8poc-sathya-stolon-lz6ws                                         0/1       Completed   0          1m
k8poc-sathya-stolon-proxy-64dfb7b59-7jmx8                         1/1       Running     0          1m
k8poc-sathya-stolon-proxy-64dfb7b59-vnt7w                         1/1       Running     0          1m
k8poc-sathya-stolon-sentinel-59dc875688-2gdxw                     1/1       Running     0          1m
k8poc-sathya-stolon-sentinel-59dc875688-97r6d                     1/1       Running     0          1m
k8poc-sathya-stolon-sentinel-59dc875688-rmrpr                     1/1       Running     0          1m

Did you try reinstalling the chart from scratch?
Did helm install actually succeeded?
Did you previously install anything using helm? maybe it's misconfigured

sathyu · 2019-02-06T19:33:38Z

Hello again,
Now stolon-sentinel is up and running but logged below error at pod level:
2019-02-06T19:02:20.069Z INFO cmd/sentinel.go:1962 sentinel uid {"uid": "70089ff6"}
2019-02-06T19:02:20.145Z INFO cmd/sentinel.go:80 Trying to acquire sentinels leadership
ERROR: logging before flag.Parse: I0206 19:02:20.145433 1 leaderelection.go:174] attempting to acquire leader lease...
ERROR: logging before flag.Parse: E0206 19:02:20.152184 1 leaderelection.go:224] error retrieving resource lock k8poc-sathya/stolon-cluster-kube-stolon
: configmaps "stolon-cluster-kube-stolon" is forbidden: User "system:serviceaccount:k8poc-sathya:stolon-sa" cannot get configmaps in the namespace "k8poc-sat
hya"
2019-02-06T19:02:20.162Z ERROR cmd/sentinel.go:1815 error retrieving cluster data {"error": "failed to get latest version of configmap: configm
aps "stolon-cluster-kube-stolon" is forbidden: User "system:serviceaccount:k8poc-sathya:stolon-sa" cannot get configmaps in the namespace "k8poc-sathya
""}
AND
stolok-keeper is crashlooping with error:
2019-02-06T19:20:34.151Z FATAL cmd/keeper.go:117 cannot get current user: cannot detect current user.

For me, role.yaml (as provided ) did not work , it errors out. So I changed it like below and ran fine.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: k8poc-sathya
name: stolon
rules:

apiGroups: ["extensions", "apps"]
resources: ["deployments", "replicasets"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
Role-binding:
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
name: rb-stolon
subjects:
kind: ServiceAccount
name: stolon-sa
namespace: k8poc-sathya
roleRef:
kind: Role
name: stolon
apiGroup: rbac.authorization.k8s.io

As always , your help is appreciated.

lwolf · 2019-02-08T19:36:24Z

so, it seems that you resolved your problem and the issue could be closed?

I'll keep in mind that additional roles might be required. But for now I can't reproduce it.
Official stolon example does not have this as well - https://github.com/sorintlab/stolon/blob/master/examples/kubernetes/role.yaml

sathyu · 2019-02-18T17:07:37Z

Hello
I did resolved the error I was getting during helm install. Now, I am getting below error when starting
stolon-keeper pods. To again emphasize, I have to run this as cluster non-admin user.
I have made some changes to keeper.yaml like below
#chown stolon:stolon $STOLON_DATA
exec stolon-keeper --data-dir $STOLON_DATA
#exec gosu stolon stolon-keeper --data-dir $STOLON_DATA
Both chown and gosu will not work for me, errors out.
let me know how can I fix this error.
Thanks
"cannot get current user: cannot detect current user"

lwolf changed the title ~~leaderelection error on k8~~ User cannot get configmaps in the namespace Jan 29, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

User cannot get configmaps in the namespace #27

User cannot get configmaps in the namespace #27

sathyu commented Jan 29, 2019

lwolf commented Jan 29, 2019 •

edited

Loading

sathyu commented Feb 1, 2019

lwolf commented Feb 3, 2019

sathyu commented Feb 4, 2019 •

edited by lwolf

Loading

lwolf commented Feb 5, 2019

sathyu commented Feb 6, 2019

lwolf commented Feb 8, 2019

sathyu commented Feb 18, 2019

User cannot get configmaps in the namespace #27

User cannot get configmaps in the namespace #27

Comments

sathyu commented Jan 29, 2019

lwolf commented Jan 29, 2019 • edited Loading

sathyu commented Feb 1, 2019

lwolf commented Feb 3, 2019

sathyu commented Feb 4, 2019 • edited by lwolf Loading

lwolf commented Feb 5, 2019

sathyu commented Feb 6, 2019

lwolf commented Feb 8, 2019

sathyu commented Feb 18, 2019

lwolf commented Jan 29, 2019 •

edited

Loading

sathyu commented Feb 4, 2019 •

edited by lwolf

Loading