# Error after restarting the elasticsearch pod managed by ECK

**URL:** <https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935>\
**Category:** Elastic Cloud on Kubernetes (ECK)\
**Created:** [November 19, 2020, 7:10am UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935 "2020-11-19T07:10:25Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vineeth\_varma](https://avatars.discourse-cdn.com/v4/letter/v/45deac/32.png) [@Vineeth\_varma](https://discuss.elastic.co/u/Vineeth_varma)\
**Post date:** [November 19, 2020, 7:10am UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/1 "2020-11-19T07:10:25Z")

</div>

I am currently using ECK version 1.2 and Elasticsearch version 7.10.0 managed by ECK. I have restarted one of the elasticsearch pods and I am getting the below errors after pod has restarted.

kubectl logs -n eo elastic-es-ingest-data-0  
{"type": "server", "timestamp": "2020-11-19T06:24:33,404Z", "level": "ERROR", "component": "o.e.b.ElasticsearchUncaughtExceptionHandler", "cluster.name": "elastic", "node.name": "elastic-es-ingest-data-0", "message": "uncaught exception in thread [main]",  
"stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Unable to access 'path.data' (/usr/share/elasticsearch/data)",  
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:174) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) ~[elasticsearch-cli-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"Caused by: java.lang.IllegalStateException: Unable to access 'path.data' (/usr/share/elasticsearch/data)",  
"at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:72) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:306) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:253) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Security.configure(Security.java:121) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:222) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"... 6 more",  
"Caused by: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data",  
"at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]",  
"at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]",  
"at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]",  
"at sun.nio.fs.UnixFileSystemProvider.checkAccess(UnixFileSystemProvider.java:312) ~[?:?]",  
"at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:394) ~[elasticsearch-7.10.0.jar:7.10.0]",  
uncaught exception in thread [main]  
"at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:70) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:306) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:253) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Security.configure(Security.java:121) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:222) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.10.0.jar:7.10.0]",  
"... 6 more"] }

I am currently doing all this for testing purposes, once everything is working fine then I will implement this in production.  
So I am desperately looking for a solution for this issue  
Answer would be really appreciated.

Thank you.

---

<div class="post-metadata">

**Author:** ![charith-elastic](https://avatars.discourse-cdn.com/v4/letter/c/3ec8ea/32.png) [@charith-elastic](https://discuss.elastic.co/u/charith-elastic)\
**Post date:** [November 19, 2020, 9:07am UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/2 "2020-11-19T09:07:15Z")

</div>

Was the cluster working correctly until you restarted the pod? What is the volume configuration you are using? Please provide your Elasticsearch manifest and information about your Kubernetes cluster -- such as the version, flavour (GKE, EKS, On-prem etc.) and whether you have pod security policies configured.

---

<div class="post-metadata">

**Author:** ![Vineeth\_varma](https://avatars.discourse-cdn.com/v4/letter/v/45deac/32.png) [@Vineeth\_varma](https://discuss.elastic.co/u/Vineeth_varma)\
**Post date:** [November 19, 2020, 10:12am UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/3 "2020-11-19T10:12:14Z")

</div>

Hi charith,

- The cluster is configured currectly and it is working fine before restart.

- Kubernetes version - 1.13

- Volume I am using mapr fs, (pvc)

- On-prem

- No pod security polocies configured.

apiVersion: [elasticsearch.k8s.elastic.co/v1](http://elasticsearch.k8s.elastic.co/v1)  
kind: Elasticsearch  
spec:  
auth: {}  
http:  
service:  
metadata:  
creationTimestamp: null  
spec:  
ports:  
- name: https  
port: 9200  
protocol: TCP  
targetPort: 9200  
type: LoadBalancer  
tls:  
certificate: {}  
nodeSets:

- config:  
node.data: false  
node.ingest: false  
node.master: true  
count: 3  
name: master  
podTemplate:  
metadata:  
creationTimestamp: null  
spec:  
containers:  
- name: elasticsearch  
resources: {}  
volumeMounts:  
- mountPath: /es-snaps  
name: es-snap-repo-vol  
initContainers:  
- command:  
- sh  
- -c  
- sysctl -w vm.max\_map\_count=262144  
name: sysctl  
resources: {}  
securityContext:  
privileged: true  
volumes:  
- name: es-snap-repo-vol  
persistentVolumeClaim:  
claimName: es-snap-repo-claim  
volumeClaimTemplates:
  - metadata:  
creationTimestamp: null  
name: elasticsearch-data  
spec:  
accessModes:
    - ReadWriteOnce  
resources:  
requests:  
storage: 10Gi  
storageClassName: sc-eo  
status: {}

- config:  
node.data: true  
node.ingest: true  
node.master: false  
path.repo:
  - /es-snaps  
count: 3  
name: ingest-data  
podTemplate:  
metadata:  
creationTimestamp: null  
spec:  
affinity:  
podAntiAffinity:  
preferredDuringSchedulingIgnoredDuringExecution:  
- podAffinityTerm:  
labelSelector:  
matchLabels:  
[elasticsearch.k8s.elastic.co/cluster-name:](http://elasticsearch.k8s.elastic.co/cluster-name:) quickstart  
topologyKey: [kubernetes.io/hostname](http://kubernetes.io/hostname)  
weight: 100  
containers:
    - name: elasticsearch  
resources: {}  
volumeMounts:
      - mountPath: /es-snaps  
name: es-snap-repo-vol  
initContainers:

    - command:
      - sh
      - -c
      - sysctl -w vm.max\_map\_count=262144  
name: sysctl  
resources: {}  
securityContext:  
privileged: true  
volumes:

    - name: es-snap-repo-vol  
persistentVolumeClaim:  
claimName: es-snap-repo-claim  
volumeClaimTemplates:

  - metadata:  
creationTimestamp: null  
name: elasticsearch-data  
spec:  
accessModes:
    - ReadWriteOnce  
resources:  
requests:  
storage: 50Gi  
storageClassName: sc-eo  
status: {}  
transport:  
service:  
metadata:  
creationTimestamp: null  
spec: {}  
updateStrategy:  
changeBudget: {}  
version: 7.10.0

---

<div class="post-metadata">

**Author:** ![Vineeth\_varma](https://avatars.discourse-cdn.com/v4/letter/v/45deac/32.png) [@Vineeth\_varma](https://discuss.elastic.co/u/Vineeth_varma)\
**Post date:** [November 19, 2020, 10:19am UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/4 "2020-11-19T10:19:12Z")

</div>

Sorry it is not coming nicely the Yaml file

---

<div class="post-metadata">

**Author:** ![charith-elastic](https://avatars.discourse-cdn.com/v4/letter/c/3ec8ea/32.png) [@charith-elastic](https://discuss.elastic.co/u/charith-elastic)\
**Post date:** [November 19, 2020, 10:36am UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/5 "2020-11-19T10:36:48Z")

</div>

You can use Markdown-style triple backticks (```) to enclose the YAML. It is difficult to read in its current form.

Also, please include the output from `kubectl describe elastic-es-ingest-data-0 -n eo` as well.

---

<div class="post-metadata">

**Author:** ![Vineeth\_varma](https://avatars.discourse-cdn.com/v4/letter/v/45deac/32.png) [@Vineeth\_varma](https://discuss.elastic.co/u/Vineeth_varma)\
**Post date:** [November 19, 2020, 12:15pm UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/6 "2020-11-19T12:15:54Z")

</div>

Yes I am trying but I wont be able to post the yaml in correct format.  
While describing the pod I am getting  
kubectl describe -n eo elastic-es-ingest-data-0  
error: the server doesn't have a resource type "elastic-es-ingest-data-0"  
because it havent even started.

---

<div class="post-metadata">

**Author:** ![charith-elastic](https://avatars.discourse-cdn.com/v4/letter/c/3ec8ea/32.png) [@charith-elastic](https://discuss.elastic.co/u/charith-elastic)\
**Post date:** [November 19, 2020, 1:16pm UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/7 "2020-11-19T13:16:49Z")

</div>

Sorry I made a mistake with the command, it should be: `kubectl describe pod elastic-es-ingest-data-0 -n eo`.

Not sure I understand why you can't format the YAML though. It really helps to see YAML in the correct format because spacing is significant.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 8:17am UTC](https://discuss.elastic.co/t/error-after-restarting-the-elasticsearch-pod-managed-by-eck/255935/8 "2022-11-04T08:17:59Z")

</div>


