Elasticsearch cloud ignoring bucket permissions

Elastic Orchestration Elastic Cloud on Kubernetes (ECK)
1

I have created an elasticsearch cluster according to:

using the following yaml

cat << 'EOF' > ~/eck.yaml
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
 name: quickstart
spec:
 version: 7.6.1
 secureSettings:
 - secretName: gcs-credentials
 nodeSets:
 - name: default
   count: 3
   config:
     node.master: true
     node.data: true
     node.ingest: true
     node.store.allow_mmap: false
   podTemplate:
     spec:
       initContainers:
       - name: install-plugins
         env:
           - name: ES_PATH_CONF
             value: /etc/elasticsearch
         command:
         - sh
         - -c
         - |
           bin/elasticsearch-plugin install --batch repository-gcs ingest-attachment
---
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
 name: kibana-sample
spec:
 version: 7.6.1
 count: 1
 elasticsearchRef:
   name: quickstart
---
apiVersion: apm.k8s.elastic.co/v1
kind: ApmServer
metadata:
 name: apm-server-sample
spec:
 version: 7.6.1
 count: 1
 elasticsearchRef:
   name: quickstart
EOF

kubectl apply -f eck.yaml

the service account has storage admin access on the bucket to read and write but i get the following error when creating a snapshot using google cloud storage, client: default, bucket :ecksnapshotraindrop, cannot verify

{
  "error": {
    "root_cause": [
      {
        "type": "blob_store_exception",
        "reason": "Unable to check if bucket [ecksnapshotraindrop] exists"
      }
    ],
    "type": "repository_exception",
    "reason": "[examplerepo] cannot create blob store",
    "caused_by": {
      "type": "blob_store_exception",
      "reason": "Unable to check if bucket [ecksnapshotraindrop] exists",
      "caused_by": {
        "type": "access_control_exception",
        "reason": "access denied (\"java.lang.RuntimePermission\" \"accessDeclaredMembers\")"
      }
    }
  },
  "status": 500
}

according to:

I must set ES_PATH_CONF to /etc/elasticsearch
I attempted to set this in the container spec above, but it did not work. How do I set ES_PATH_CONF in gcloud?

2

This is not required. The steps described here should help you to use GCS for you snapshots.
I would double check that the GCP service account has the relevant permissions to access the bucket and also check the content of gcs.client.default.credentials_file in the gcs-credentials Secret .

See also my answer here if you want to try to rely on Workload Identity.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.