# Support for ECK with a PSP **URL:** \ **Category:** Elastic Cloud on Kubernetes (ECK)\ **Tags:** docker\ **Created:** [September 22, 2020, 6:14am UTC](https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454 "2020-09-22T06:14:28Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [September 22, 2020, 6:14am UTC](https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454/1 "2020-09-22T06:14:28Z")
I am trying to use ECK with PSP enabled in my self-hosted kubernetes cluster. It does not seem to work, and it has a weird deployment method of using a stateful set to deploy additional privileged containers. So, what permissions should I give for ECK to work and to which service account? There seems to be an issue open for this. Would prefer to have a sample PSP shipped with the operator as this would make sense for users who enable PSP by default during installation! > > > This could be part or #2785 or a standalone sample configuration to illustrate a secure ECK setup when users have turned... The existing PSP used are mentioned on the Github issue. I finally had to allow all serviceaccounts to create a privileged pod in the Elasticsearch namespace which was definitely an overkill and a **security risk**. Am I missing something here? ---
**Author:** ![dkow](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dkow/32/47340_2.png) [@dkow](https://discuss.elastic.co/u/dkow)\ **Post date:** [September 22, 2020, 8:32am UTC](https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454/2 "2020-09-22T08:32:06Z")
Hi @NerdSec, thanks for moving to Discuss. When I glanced over Calico I was under an impression that it's a k8s distribution and I worded my response like it mattered. Apologies! To reiterate - ECK operator does not create Pods directly (privileged or not). Instead, we create a StatefulSet resource. There is a built-in StatefulSet controller in k8s that creates Pods based on that StatefulSet resource. This is important, because it means that `elastic-operator` is not the right Service Account to give `use` permission for a PSP. So what's the right way? Well, there are [two](https://kubernetes.io/docs/concepts/policy/pod-security-policy/#authorizing-policies): - give the `use` permission to the StatefulSet controller - have the Pods use `serviceAccount` that has the right `use` permission As you correctly noted, the first one is more permissive than needed. I suggested that one as I felt it can help unblocking and verifying if that's really the issue. I think that my response might've lacked context 🙂 The second way is preferred, but requires to create a Service Account, the right Binding and providing it in the Elasticsearch manifest `podTemplate`. Our E2E test suite runs on PSP enabled clusters and the PSP is in our [repo](https://github.com/elastic/cloud-on-k8s/blob/master/config/dev/elastic-psp.yaml), if you'd like to take a look. Btw, where you able to check `kubernetes.io/psp` annotation on the Pod? It should tell you which PSP was the Pod created with. Let me know if the above allowed you to unblock, I'll be happy to help further if not. ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [September 22, 2020, 12:13pm UTC](https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454/3 "2020-09-22T12:13:09Z")
Hi David, > [@dkow](#): > > Btw, where you able to check `kubernetes.io/psp` annotation on the Pod? It should tell you which PSP was the Pod created with. So, yes, by default I have allowed all authenticated users to use the `z-restricted` PSP. This is stopping the init container from being created. ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/b/ab4244d076db91a3e80e93eee57ee236e823ec4f.png) Adding a more permissive use permission for all SA in the elasticsearch namespace allows the pods to use the `z-privileged` policy, and thereby the containers get created. I will try using an appropriate role binding and provide it in the podTemplate. Thanks for clarifying this up! However, I remember trying out the BETA ECK without having to make any such changes, did something change? Why does the init container need to run as a `root` user with access to a writable `rootfs`? ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [September 22, 2020, 1:11pm UTC](https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454/4 "2020-09-22T13:11:06Z")
So, I have not created a cluster role as such, I simply allowed the `default` serviceaccount the `use` permission for my `z-privileged` PSP. The deployment happened and I can see the pods use the privileged PSP. The PSP you shared, is similar to the restricted PSP available on my cluster. Using that PSP, the pod is not allowed to load. ```auto Labels: common.k8s.elastic.co/type=elasticsearch controller-revision-hash=brb-es-default-fffd7cb5b elasticsearch.k8s.elastic.co/cluster-name=brb elasticsearch.k8s.elastic.co/config-hash=1841883488 elasticsearch.k8s.elastic.co/http-scheme=https elasticsearch.k8s.elastic.co/node-data=true elasticsearch.k8s.elastic.co/node-ingest=true elasticsearch.k8s.elastic.co/node-master=true elasticsearch.k8s.elastic.co/node-ml=true elasticsearch.k8s.elastic.co/statefulset-name=brb-es-default elasticsearch.k8s.elastic.co/version=7.9.1 statefulset.kubernetes.io/pod-name=brb-es-default-0 Annotations: cni.projectcalico.org/podIP: 192.168.193.59/32 cni.projectcalico.org/podIPs: 192.168.193.59/32 co.elastic.logs/module: elasticsearch container.apparmor.security.beta.kubernetes.io/elastic-internal-init-filesystem: runtime/default container.apparmor.security.beta.kubernetes.io/elasticsearch: runtime/default kubernetes.io/psp: elastic.restricted seccomp.security.alpha.kubernetes.io/pod: runtime/default ``` Events: ```auto Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 30s default-scheduler Successfully assigned elastic/cdc-es-default-0 to sidcirmkube02 Normal Pulled kubelet, sidcirmkube02 Successfully pulled image "docker.elastic.co/elasticsearch/elasticsearch:7.9.1" in 4.951464499s Normal Pulled kubelet, sidcirmkube02 Successfully pulled image "docker.elastic.co/elasticsearch/elasticsearch:7.9.1" in 3.733738989s Normal Pulling (x3 over ) kubelet, sidcirmkube02 Pulling image "docker.elastic.co/elasticsearch/elasticsearch:7.9.1" Warning Failed (x3 over ) kubelet, sidcirmkube02 Error: container has runAsNonRoot and image will run as root Normal Pulled kubelet, sidcirmkube02 Successfully pulled image "docker.elastic.co/elasticsearch/elasticsearch:7.9.1" in 3.52929003s ``` Is there a different image being used over here in the E2E environment? ---
**Author:** ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)\ **Post date:** [September 23, 2020, 6:18am UTC](https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454/5 "2020-09-23T06:18:05Z")
As you mentioned correctly, had to set the securityContext in the podTemplate. After setting the Context it worked absolutely fine. For someone who wants to do this, here are the sample yamls: ```auto # Elasticsearch apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: cdc spec: version: 7.9.1 nodeSets: - name: default count: 3 config: node.master: true node.data: true node.ingest: true node.store.allow_mmap: false volumeClaimTemplates: - metadata: name: elasticsearch-data spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi storageClassName: longhorn podTemplate: spec: securityContext: fsGroup: 1000 runAsUser: 1000 runAsGroup: 1000 ``` ```auto # Kibana apiVersion: kibana.k8s.elastic.co/v1 kind: Kibana metadata: name: cdc spec: version: 7.9.1 count: 1 elasticsearchRef: name: cdc http: service: spec: ports: - protocol: TCP port: 443 targetPort: 5601 type: LoadBalancer loadBalancerIP: 10.31.24.22 tls: selfSignedCertificate: subjectAltNames: - ip: 10.31.24.22 - dns: kibana.xio.com podTemplate: spec: securityContext: fsGroup: 1000 runAsUser: 1000 runAsGroup: 1000 ``` @dkow Thanks for the inputs! By any chance can we add this to the ECK documentation? This would be easier for someone like me who is new to Kubernetes & ECK to start off in a much more secure manner. Also, why is this not included in the ECK operator? Shouldn't this be like the default setting? The way I see it, specifying the `USER` in the Dockerfile could also solve this? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 8:13am UTC](https://discuss.elastic.co/t/support-for-eck-with-a-psp/249454/6 "2022-11-04T08:13:40Z")