# Elasticsearch nodes keep restarting on Kubernetes

**URL:** <https://discuss.elastic.co/t/elasticsearch-nodes-keep-restarting-on-kubernetes/125936>\
**Category:** Elasticsearch\
**Created:** [March 28, 2018, 12:47pm UTC](https://discuss.elastic.co/t/elasticsearch-nodes-keep-restarting-on-kubernetes/125936 "2018-03-28T12:47:34Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mightwork](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mightwork/32/29311_2.png) [@mightwork](https://discuss.elastic.co/u/mightwork)\
**Post date:** [March 28, 2018, 12:47pm UTC](https://discuss.elastic.co/t/elasticsearch-nodes-keep-restarting-on-kubernetes/125936/1 "2018-03-28T12:47:34Z")

</div>

Hi,

I have a 8 node cluster setup, with 3 master, 2 ingest and 3 data nodes running on Kubernetes(1.9.2). I am using a custom build image based on the official 6.2.2-x-pack image. The Dockerfile is below:

```
FROM docker.elastic.co/elasticsearch/elasticsearch-basic:6.2.2

VOLUME ["/usr/share/elasticsearch/data"]

ENV ES_JAVA_OPTS "-Xms5g -Xmx5g"
ENV CLUSTER_NAME elasticsearch-default
ENV NODE_MASTER true
ENV NODE_DATA true
ENV NODE_INGEST true
ENV HTTP_ENABLE true
ENV NETWORK_HOST _site_
ENV HTTP_CORS_ENABLE true
ENV HTTP_CORS_ALLOW_ORIGIN *
ENV NUMBER_OF_MASTERS 1
ENV MAX_LOCAL_STORAGE_NODES 1
ENV SHARD_ALLOCATION_AWARENESS ""
ENV SHARD_ALLOCATION_AWARENESS_ATTR ""
ENV MEMORY_LOCK true
ENV REPO_LOCATIONS []

USER root
COPY config/jvm.options /usr/share/elasticsearch/config/
COPY config/elasticsearch.yml /usr/share/elasticsearch/config/
COPY config/log4j2.properties /usr/share/elasticsearch/config/
COPY run.sh .

RUN chown elasticsearch:elasticsearch /usr/share/elasticsearch/config/jvm.options \
/usr/share/elasticsearch/config/elasticsearch.yml \
/usr/share/elasticsearch/config/log4j2.properties \
run.sh && chmod 0750 run.sh 

CMD ["/bin/sh", "run.sh"]

```

run.sh -

```
#!/bin/sh
set -ex

ulimit -n 65536
ulimit -u 4096
ulimit -l unlimited

chown -R elasticsearch:elasticsearch /usr/share/elasticsearch
chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/data

su elasticsearch /usr/local/bin/docker-entrypoint.sh /usr/share/elasticsearch/bin/elasticsearch

```

elasticsearch.yml

```
cluster:
  name: ${CLUSTER_NAME}

node:
  master: ${NODE_MASTER}
  name: ${NODE_NAME}
  data: ${NODE_DATA}
  ingest: ${NODE_INGEST}
  ml: false
  max_local_storage_nodes: ${MAX_LOCAL_STORAGE_NODES}

processors: ${PROCESSORS:1}

network.host: ${NETWORK_HOST}

path:
  data: /data/data
  logs: /data/log
  repo: ${REPO_LOCATIONS}

bootstrap:
  memory_lock: ${MEMORY_LOCK}

http:
  enabled: ${HTTP_ENABLE}
  compression: true
  cors:
    enabled: ${HTTP_CORS_ENABLE}
    allow-origin: ${HTTP_CORS_ALLOW_ORIGIN}

discovery:
  zen:
    minimum_master_nodes: ${NUMBER_OF_MASTERS}
    ping.unicast.hosts: elasticsearch-discovery

xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.ml.enabled: false

xpack.monitoring.exporters.my_local:
  type: local
  use_ingest: false

```

The nodes just restart without any error message and it requires deleting the pod or deployment multiple times before they finally stabilize.

Logs from master and ingest nodes: [https://pastebin.com/Qc1sWMaG](https://pastebin.com/Qc1sWMaG)

Any clue as to why this is happening and how to fix this?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 25, 2018, 12:47pm UTC](https://discuss.elastic.co/t/elasticsearch-nodes-keep-restarting-on-kubernetes/125936/2 "2018-04-25T12:47:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.