# Issue with ingesting data and disk size

**URL:** <https://discuss.elastic.co/t/issue-with-ingesting-data-and-disk-size/336087>\
**Category:** Elasticsearch\
**Created:** [June 15, 2023, 11:36am UTC](https://discuss.elastic.co/t/issue-with-ingesting-data-and-disk-size/336087 "2023-06-15T11:36:31Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![ANUBHAV\_GUPTA](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@ANUBHAV\_GUPTA](https://discuss.elastic.co/u/ANUBHAV_GUPTA)\
**Post date:** [June 15, 2023, 11:36am UTC](https://discuss.elastic.co/t/issue-with-ingesting-data-and-disk-size/336087/1 "2023-06-15T11:36:31Z")

</div>

I am facing a very weird issue.  
I tried uploading 30 GB of csv data in Elasticsearch using python client.

The below is the disk usage when I quit ingestion:-

```auto
shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
     1 28gb 30.3gb 1gb 31.3gb 96 10.0.0.2 10.0.0.2 113b51464987

```

And after couple of min when I saw it was the following:-

```auto
shards disk.indices disk.used disk.avail disk.total disk.percent host ip node
     1 22.9gb 24.7gb 6.5gb 31.3gb 78 10.0.0.2 10.0.0.2 113b51464987

```

Can anyone please elaborate on this?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [June 19, 2023, 8:26am UTC](https://discuss.elastic.co/t/issue-with-ingesting-data-and-disk-size/336087/2 "2023-06-19T08:26:36Z")

</div>

It's likely because you had segments being merged in the background which has brought the disk space down.

---

<div class="post-metadata">

**Author:** ![ANUBHAV\_GUPTA](https://avatars.discourse-cdn.com/v4/letter/a/7ea924/32.png) [@ANUBHAV\_GUPTA](https://discuss.elastic.co/u/ANUBHAV_GUPTA)\
**Post date:** [June 19, 2023, 9:07am UTC](https://discuss.elastic.co/t/issue-with-ingesting-data-and-disk-size/336087/3 "2023-06-19T09:07:04Z")

</div>

Thanks for the reply!  
I am facing this issue everytime while ingesting.  
When I am about to complete ingesting like say 30GB of data, the next minute the total storage comes back to 25GB or even less.

Is this the normal behavior?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [June 19, 2023, 9:23am UTC](https://discuss.elastic.co/t/issue-with-ingesting-data-and-disk-size/336087/4 "2023-06-19T09:23:29Z")

</div>

> [@ANUBHAV\_GUPTA](#):
>
> Is this the normal behavior?

Yes. Elasticsearch creates immutable segments during indexing. These are periodically merged/combined into larger segments. During a merge new segments will be created while before the old ones are removed, so disk space can increase and contract a bit over time. This is likely to occur after your final segment has been created and indexing has completed, which is what you are seeing.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 17, 2023, 9:24am UTC](https://discuss.elastic.co/t/issue-with-ingesting-data-and-disk-size/336087/5 "2023-07-17T09:24:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
