# Elasticsearch Docker Container filling up disk

**URL:** <https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904>\
**Category:** Elasticsearch\
**Created:** [October 3, 2018, 2:16pm UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904 "2018-10-03T14:16:44Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\
**Post date:** [October 3, 2018, 2:16pm UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904/1 "2018-10-03T14:16:45Z")

</div>

I've been doing some experimentation and development with Elasticsearch and Kibana running in the official docker containers. I've got the ElasticSearch /usr/share/elasticsearch/data mapped to a 1.5TB drive which seems to be automatically maintaining the disk space to 95%.

However, for some reason my / drive (which also handles the /var/lib/docker/overlay2/ container storage) has filled up.

How can I figure out what inside the docker containers (either Kibana or Elasticsearch) is filling up my / drive? Should I relocate my default /var/lib/docker to the external /data drive?

My docker-compose configs are like this:

```
- name: Build Elasticsearch Docker Image
  docker_service:
    project_name: elasticsearch
    definition:
      version: '3'
      services:
        elasticsearch:
          image: docker.elastic.co/elasticsearch/elasticsearch:{{ elastic_version }}
          container_name: elasticsearch
          restart_policy:
            condition: on-failure
            delay: 5s
            max_attempts: 3
            window: 120s
          environment:
            - bootstrap.memory_lock=true
            - "ELASTIC_PASSWORD=changeme"
            - "ES_JAVA_OPTS=-Xms16g -Xmx16g"
            - xpack.security.enabled=false
          ulimits:
            memlock:
              soft: -1
              hard: -1
            nofile:
              soft: 65536
              hard: 65536
          volumes:
            - /data/elasticsearch/data:/usr/share/elasticsearch/data
            - /var/log/elasticsearch:/var/log/elasticsearch
          ports:
            - 9200:9200
          networks: ['stack']

      networks: {stack: {}}
 
- name: Build Kibana Docker Image
  docker_service:
    project_name: kibana
    definition:
      version: '3'
      services:
        kibana:
          image: docker.elastic.co/kibana/kibana:{{ kibana_version }}
          container_name: kibana
          restart_policy:
            condition: on-failure
            delay: 5s
            max_attempts: 3
            window: 120s
          environment:
            - ELASTICSEARCH_URL=http://localhost:9200
          ports:
            - 5601:5601
          networks: ['stack']

      networks: {stack: {}}
```

---

<div class="post-metadata">

**Author:** ![bardie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bardie/32/36173_2.png) [@bardie](https://discuss.elastic.co/u/bardie)\
**Post date:** [October 5, 2018, 11:05am UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904/2 "2018-10-05T11:05:10Z")

</div>

How much space do you have on /root?

Are you able to send a screenshot after running the following command:

`df -h`

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [October 5, 2018, 11:34am UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904/3 "2018-10-05T11:34:41Z")

</div>

> [@sgreszcz](#):
>
> ```auto
> - /var/log/elasticsearch:/var/log/elasticsearch
> 
> ```

Are copious logs being written to `/`? A healthy Elasticsearch cluster writes very few logs, so this might indicate a problem that needs addressing.

---

<div class="post-metadata">

**Author:** ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\
**Post date:** [October 8, 2018, 5:48pm UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904/4 "2018-10-08T17:48:17Z")

</div>

In the end, I couldn't even shell into the docker container due to disk full. I ended up rm'ing the docker containers, moving the /var/lib/docker directory to my larger "/data" drive, and doing another docker-compose to pull the new images. Hopefully this works ok with the extra storage headroom.

```auto
sudo df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/elk--alln--001--vg-root 39G 39G 0 100% /
/dev/sda1 472M 132M 316M 30% /boot
/dev/mapper/data-data1 1.5T 1.5T 79G 95% /data
overlay 39G 39G 0 100% /var/lib/docker/overlay2/1b9b77164ac19e291491b12151c9fe26a1551d72b3c66c015b43bcd1ff418770/merged

```

---

<div class="post-metadata">

**Author:** ![sgreszcz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sgreszcz/32/46322_2.png) [@sgreszcz](https://discuss.elastic.co/u/sgreszcz)\
**Post date:** [October 8, 2018, 5:49pm UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904/5 "2018-10-08T17:49:18Z")

</div>

I linked /var/lib/docker to /data/docker (a bigger drive) then re-pulled the elasticsearch and kibana docker images. Hopefully this helps not kill my / drive next time.

---

<div class="post-metadata">

**Author:** ![bardie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bardie/32/36173_2.png) [@bardie](https://discuss.elastic.co/u/bardie)\
**Post date:** [October 14, 2018, 1:07pm UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904/6 "2018-10-14T13:07:04Z")

</div>

> [@sgreszcz](#):
>
> 316M 30% /boot /dev/mapper/data-data1 1.5T 1.5T 79G 95% /data

This will work, i usually have the same problem so i usually assign 200gb (Running 10+ docker images) for my /var partition or just change the default docker directory

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 11, 2018, 1:22pm UTC](https://discuss.elastic.co/t/elasticsearch-docker-container-filling-up-disk/150904/7 "2018-11-11T13:22:02Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
