# Docker Hard Disk Image File Being Too Large

**URL:** https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443
**Category:** Logstash
**Tags:** docker
**Created:** [November 18, 2023, 6:30am UTC](https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443 "2023-11-18T06:30:05Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![Ethan777100](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethan777100/32/70366_2.png) [@Ethan777100](https://discuss.elastic.co/u/Ethan777100)
#### Post date: [November 18, 2023, 6:30am UTC](https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443/1 "2023-11-18T06:30:05Z")

</div>

Its now killing my computer space. I'm in the midst of ingesting 537 csv files (total 10.7GB) into elasticsearch.

`C:\Users\ethan\AppData\Local\Docker\wsl\data`

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/0/00c88b826e208c7e2df42939226511eefd07fd71.png)

I'm not sure why it became so big.

![image](https://us1.discourse-cdn.com/elastic/original/3X/2/a/2a5a15769ab285b86bc8dc6714b58cab63c59e42.png)

---

<div class="post-metadata">

### Author: ![Ethan777100](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethan777100/32/70366_2.png) [@Ethan777100](https://discuss.elastic.co/u/Ethan777100)
#### Post date: [November 20, 2023, 9:43am UTC](https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443/2 "2023-11-20T09:43:41Z")

</div>

TYPE TOTAL ACTIVE SIZE RECLAIMABLE  
Images 3 2 3.057GB 766.5MB (25%)  
Containers 3 2 1.32GB 0B (0%)  
Local Volumes 3 3 2.02GB 0B (0%)  
Build Cache 0 0 0B 0B

My "C:\Users\ethan\AppData\Local\Docker\wsl\data\ext4.vhdx" being 25.6GB huge when I'm merely ingesting 492MB worth of csv files?

I checked my configs in logstash.yml and logstash.conf.

I bind mounted these files to logstash to be piped in.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/a/8a754adb89a2522db5c3a0916f5107d19f9738d7.png)

I have tweaked things. Now I am only ingested 1 month worth of csv files to investigate.

I did docker system df and i see that my Local Volumes are growing in sizze. Nothing can be relcaimed in space.

Within logstash, I can confirm bind mounts to my csv files.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/a/7a6999400b3c76be9c1783b062e8949b734de5f3.png)

However, I searched within the ES file directory. Can't find anything big where my datalake resides.

So from 8.5GB -\> 14.6GB and counting.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/4/640c9a2f7235c39225a77d01b65cde68d845e4d4.png)

========================================

I have tried shutting down docker and killing off `vmmem.wsl` so that I can use `diskpart` to `compact vdisk`. I could cut the file size fo 8.5GB.

However when docker restarts, the file grows again.

THis is not good. Im not sure what else to troubleshoot. Pls help me.

---

<div class="post-metadata">

### Author: ![Ethan777100](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethan777100/32/70366_2.png) [@Ethan777100](https://discuss.elastic.co/u/Ethan777100)
#### Post date: [November 20, 2023, 6:19pm UTC](https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443/3 "2023-11-20T18:19:30Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/b/4/b489bd85d4cc19cefa7b48d26978faaca6da8955.png)

At 1 million rows ingested, the disk increased a little slower from 10.5 GB -\> 13.5 GB

I did some improvements to streamline how the mapping is done on the columns.

Disk is growing still...but it does feel growing a little slower.

Not sure how much more can I improve on things to make the Disk grow minimally.

Now I am only ingesting 30 days of csv file. Each day about 137000 rows of data.

Eventually I want to ingest 18 months worth of csv files of 137000 rows of Data.

I only have about 200GB of free space to work with. I dont want to burst my file space budget.

I might even, have more csv files to ingest in future - of other data sets.

**PS: My files ingestion stopped midway. But the disk is still growing now at 25 GB! This is astonishing!**

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/4/c46c93873663b4824418c9b2aa654a6d46d3a279.png)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 18, 2023, 6:19pm UTC](https://discuss.elastic.co/t/docker-hard-disk-image-file-being-too-large/347443/4 "2023-12-18T18:19:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
