# Docker ELK Stack, changed compose, now shards missing

**URL:** <https://discuss.elastic.co/t/docker-elk-stack-changed-compose-now-shards-missing/103429>\
**Category:** Elasticsearch\
**Created:** [October 10, 2017, 7:48pm UTC](https://discuss.elastic.co/t/docker-elk-stack-changed-compose-now-shards-missing/103429 "2017-10-10T19:48:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Brandon\_Martin](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Brandon\_Martin](https://discuss.elastic.co/u/Brandon_Martin)\
**Post date:** [October 10, 2017, 7:48pm UTC](https://discuss.elastic.co/t/docker-elk-stack-changed-compose-now-shards-missing/103429/1 "2017-10-10T19:48:01Z")

</div>

I'm using elasticsearch docker containers for my nodes.  
I began experimenting with Scripts, and needed to add a volume. Here's what my docker-compose now contains

```auto
elasticsearch:
    build: elasticsearch/
    container_name: elasticsearch
    volumes:
      - ./elasticsearch/config/elasticsearch.yml:/conf/elasticsearch.yml
      - ./elasticsearch/config/scripts:/conf/scripts
      - /data/elasticsearch:/usr/share/elasticsearch/data:rw
    environment:
      ES_JAVA_OPTS: "-Xmx4g -Xms4g"
      TYPE: MASTER
      UNICAST_HOSTS: elasticsearch,elasticsearch_3,elasticsearch_2,elasticsearch_4,elasticsearch_1
      MIN_MASTERS: 1
      PLUGINS: x-pack
    networks:
      - elk
    restart: always

```

Using data volumes, my belief was I could update the docker containers without losing data. After adding the scripts folder, compose re-created the containers, and now my shards have gone missing.  
I can still see the data in the folder (/data/elasticsearch1/nodes/0/, 130gb), but my shards have reset and elasticsearch is not using my previous data.  
Is there a way to tell elasticsearch to rescan the data directory and re-allocate those shards from before? Or am I using the data volume incorrectly?

---

<div class="post-metadata">

**Author:** ![Brandon\_Martin](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Brandon\_Martin](https://discuss.elastic.co/u/Brandon_Martin)\
**Post date:** [October 10, 2017, 8:03pm UTC](https://discuss.elastic.co/t/docker-elk-stack-changed-compose-now-shards-missing/103429/2 "2017-10-10T20:03:25Z")

</div>

I'm reading [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/_path_to_data_on_disk.html), "Upon startup, Elasticsearch will check to see if the cluster folder exists and has data, and will read from it if necessary."  
It doesn't seem my elasticsearch nodes are doing as such.

---

<div class="post-metadata">

**Author:** ![Brandon\_Martin](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Brandon\_Martin](https://discuss.elastic.co/u/Brandon_Martin)\
**Post date:** [October 10, 2017, 8:29pm UTC](https://discuss.elastic.co/t/docker-elk-stack-changed-compose-now-shards-missing/103429/3 "2017-10-10T20:29:29Z")

</div>

More reading [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#node.name)  
Most probable cause is the node IDs changed. I've added the [node.name](http://node.name) parameter in my elasticsearch.yml files. This way they won't change.

I still am not sure how to force elasticsearch to check existing data. I tried renaming the \_state folder, a new was created and still didn't see old data.

---

<div class="post-metadata">

**Author:** ![Brandon\_Martin](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Brandon\_Martin](https://discuss.elastic.co/u/Brandon_Martin)\
**Post date:** [October 12, 2017, 5:26pm UTC](https://discuss.elastic.co/t/docker-elk-stack-changed-compose-now-shards-missing/103429/4 "2017-10-12T17:26:20Z")

</div>

Problem was my Dockerfile set elasticsearch to use /data for the volume and data directory. Changing my volume to /data:/data solved the issue.  
I was able to collect all indices from failed container volumes, re-assigning shards now.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 9, 2017, 5:27pm UTC](https://discuss.elastic.co/t/docker-elk-stack-changed-compose-now-shards-missing/103429/5 "2017-11-09T17:27:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.