# Elastic 8.11.3 on docker in OSX silicon has disk volume sizing issues

**URL:** <https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [December 17, 2023, 10:07pm UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522 "2023-12-17T22:07:49Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![matthal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthal/32/130095_2.png) [@matthal](https://discuss.elastic.co/u/matthal)\
**Post date:** [December 17, 2023, 10:07pm UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/1 "2023-12-17T22:07:49Z")

</div>

Trying to run elasticsearch locally for development using docker compose ([Getting started with the Elastic Stack and Docker-Compose | Elastic Blog](https://www.elastic.co/blog/getting-started-with-the-elastic-stack-and-docker-compose))

I end up getting disk pressure issues, well a warning, but it ends up meaning a dysfunctional es when you try adding indexes and data.

```bash
elasticsearch-es01-1 | {"@timestamp":"2023-12-17T21:46:42.285Z", "log.level": "WARN", "message":"high disk watermark [90%] exceeded on [8QrPMNt3TZ2KKJrckbbXOg][es01][/usr/share/elasticsearch/data] free: 14.9gb[6.4%], shards will be relocated away from this node; currently relocating away shards totalling [0] bytes; the node is expected to continue to exceed the high disk watermark when these relocations are complete", "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"elasticsearch.server","process.thread.name":"elasticsearch[es01][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.cluster.routing.allocation.DiskThresholdMonitor","elasticsearch.cluster.uuid":"Xjc0-pxlSyqehu4SKDBvfw","elasticsearch.node.id":"8QrPMNt3TZ2KKJrckbbXOg","elasticsearch.node.name":"es01","elasticsearch.cluster.name":"docker-cluster"}

```

To stop this happening you can configure the following, but that just avoids the real issue.

```bash
curl -XPUT "http://localhost:9200/_cluster/settings" \
 -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster": {
      "routing": {
        "allocation.disk.threshold_enabled": false
      }
    }
  }
}'

```

The real issue is that the docker image thinks it has used up all the space, whereas docker demon correctly says what it is actually using.

```bash
docker ps -s
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES SIZE
8b4f21221f4f docker.elastic.co/elasticsearch/elasticsearch:8.11.3 "/bin/tini -- /usr/l…" 8 minutes ago Up 8 minutes (healthy) 0.0.0.0:9200->9200/tcp, 9300/tcp elasticsearch-es01-1 246kB (virtual 797MB)
0fe87d614464 docker.elastic.co/elasticsearch/elasticsearch:8.11.3 "/bin/tini -- /usr/l…" 8 minutes ago Up 8 minutes (healthy) 9200/tcp, 9300/tcp elasticsearch-setup-1 688MB (virtual 1.49GB)

```

Looks normal, but lets look inside the running container

```bash
docker exec -it 8b4f21221f4f /bin/bash
elasticsearch@8b4f21221f4f:~$ df -h
Filesystem Size Used Avail Use% Mounted on
overlay 234G 207G 15G 94% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/vda1 234G 207G 15G 94% /etc/hosts
tmpfs 20G 0 20G 0% /sys/firmware

```

Has anyone else experienced this scenario?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 18, 2023, 1:11am UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/2 "2023-12-18T01:11:00Z")

</div>

Hi @matthal Welcome to the commuity.

> [@matthal](#):
>
> `/dev/vda1 234G 207G 15G 94% /etc/hosts`

What do you have the Virtual Disk Limit Set to?

 ![Screenshot 2023-12-17 at 5.03.04 PM](https://us1.discourse-cdn.com/elastic/original/3X/c/e/ceb3f8f3b1b4a100aa263c8d9749eee7aefd1e1e.png)

That is what is being reflected if you run

`GET /_cat/nodes/?v&h=name,du,dt,dup,hp,hc,rm,rp,r`

or

```auto
sh-5.0$ df -h
Filesystem Size Used Avail Use% Mounted on
overlay 59G 37G 19G 66% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/vda1 59G 37G 19G 66% /etc/hosts <!---- Here 
tmpfs 3.9G 0 3.9G 0% /proc/acpi
tmpfs 3.9G 0 3.9G 0% /sys/firmware

```

Note: mine is set to 64GB or as the note says  
_"Due to filesystem overhead, the real available space might be less."_

So your data is still within the Docker Virtual Disk... It is just persistent...

This is docker disk stuff...

So you need to clean up something else that is taking up space on your Docker Virtual disk or increase it.

You can read more about docker storage here...

> **[Manage data in Docker](https://docs.docker.com/storage/)**
>
> Overview of persisting data in containers

---

<div class="post-metadata">

**Author:** ![matthal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthal/32/130095_2.png) [@matthal](https://discuss.elastic.co/u/matthal)\
**Post date:** [December 18, 2023, 2:09am UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/3 "2023-12-18T02:09:05Z")

</div>

I have the following

 ![Screenshot 2023-12-18 at 2.45.36 PM](https://us1.discourse-cdn.com/elastic/original/3X/2/8/284c9b1e7b77d0db13f55bf8f4b90dc6b7d42e32.png)

```auto
curl -XGET 'http://localhost:9200/_cat/nodes/?v&h=name,du,dt,dup,hp,hc,rm,rp,r'

name du dt dup hp hc rm rp r
es01 218.6gb 233.6gb 93.59 55 283.6mb 1gb 99 cdfhilmrstw

```

```auto
df -h
Filesystem Size Used Avail Use% Mounted on
overlay 234G 207G 15G 94% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/vda1 234G 207G 15G 94% /etc/hosts
tmpfs 20G 0 20G 0% /sys/firmware

```

I am not sure how Elasticsearch is using 207 G when there is no data in this, i.e. it is a fresh install.

---

<div class="post-metadata">

**Author:** ![matthal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthal/32/130095_2.png) [@matthal](https://discuss.elastic.co/u/matthal)\
**Post date:** [December 18, 2023, 2:23am UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/4 "2023-12-18T02:23:05Z")

</div>

Here is a snapshot of the volume in docker desktop too

 ![Screenshot 2023-12-18 at 3.22.16 PM](https://us1.discourse-cdn.com/elastic/original/3X/6/c/6ccb0c3046daa4e09a509b7c308c170f4ebafa68.png)

Thank you for your help @stephenb

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [December 18, 2023, 2:25am UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/5 "2023-12-18T02:25:34Z")

</div>

> [@matthal](#):
>
> I am not sure how Elasticsearch is using 207 G when there is no data in this, i.e. it is a fresh install.

Do you have any other docker containers? This is space will be used by everything you run on Docker on your Mac, it is not Elasticsearch that is using it.

This is related to Docker, not Elasticsearch.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [December 18, 2023, 2:33am UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/6 "2023-12-18T02:33:08Z")

</div>

@matthal

as @leandrojmp said, that is the share docker virtual disk

You can run

`docker system df`

and for the details...

`docker system df -v`

to see what else is taking up space

---

<div class="post-metadata">

**Author:** ![matthal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthal/32/130095_2.png) [@matthal](https://discuss.elastic.co/u/matthal)\
**Post date:** [December 18, 2023, 3:09am UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/7 "2023-12-18T03:09:07Z")

</div>

> [@stephenb](#):
>
> docker system df -v

Thanks for the reply. This makes sense.

```auto
docker system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 27 4 45.1GB 42.88GB (95%)
Containers 5 5 688.9MB 0B (0%)
Local Volumes 21 3 101.8GB 101.8GB (99%)
Build Cache 149 0 19.36GB 19.36GB

```

I removed some of the other volumes used by containers not running and got down to the following (there are some I can't remove as they are needed)

```auto
$ docker system df      
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 27 4 45.1GB 42.88GB (95%)
Containers 5 5 688.9MB 0B (0%)
Local Volumes 10 3 48.46GB 48.46GB (99%)
Build Cache 149 0 19.36GB 19.36GB

```

This reduction is reflected in the following:

```auto
df -h (in elastic container)
Filesystem Size Used Avail Use% Mounted on
overlay 234G 156G 67G 70% /
tmpfs 64M 0 64M 0% /dev
shm 64M 0 64M 0% /dev/shm
/dev/vda1 234G 156G 67G 70% /etc/hosts
tmpfs 20G 0 20G 0% /sys/firmware

```

Which then changed the health status of the cluster from [RED] to [GREEN].

I'll try to figure out why there is another 30 GB on top of the 48.46 GB reported that df -h inside the container is seeing.

In any case, now I understand what is happening here, thank you.

regards  
Matt

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 15, 2024, 3:10am UTC](https://discuss.elastic.co/t/elastic-8-11-3-on-docker-in-osx-silicon-has-disk-volume-sizing-issues/349522/8 "2024-01-15T03:10:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.