# Space is getting filled. Need some information how to clear/re-claim the space

**URL:** https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647
**Category:** Elasticsearch
**Created:** [April 26, 2018, 11:13am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647 "2018-04-26T11:13:40Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [April 26, 2018, 11:13am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/1 "2018-04-26T11:13:41Z")

</div>

Hi ,

We have Elasticserch for monitoring our services for business production. But the space is getting filled after a while. We are facing this issue quite often. Need to know if there is a way we can claim or clear the space, which logs we can clear.

```auto
[fuseadmin@a0110pcsgmon03 ~]$ df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rhel_vg-root_lv 33G 1.9G 29G 6% /
devtmpfs 32G 0 32G 0% /dev
tmpfs 32G 0 32G 0% /dev/shm
tmpfs 32G 449M 31G 2% /run
tmpfs 32G 0 32G 0% /sys/fs/cgroup
/dev/sda1 488M 147M 306M 33% /boot
/dev/mapper/rhel_vg-home_lv 30G 160M 28G 1% /home
/dev/mapper/rhel_vg-var_lv 20G 1.3G 18G 7% /var
/dev/mapper/app_vg-app_lv 1.5T 1011G 391G 73% /app
tmpfs 6.3G 0 6.3G 0% /run/user/1009
[fuseadmin@a0110pcsgmon03 ~]$ du -sh *
4.7M httpd_rpm
0 nohup.out
106M openssl-1.0.2i
5.1M openssl-1.0.2i.tar.gz
[fuseadmin@a0110pcsgmon03 ~]$

```

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [April 26, 2018, 11:14am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/2 "2018-04-26T11:14:22Z")

</div>

it will very helpful if we can get a reply soon and also let me know you have any query,

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 26, 2018, 4:53pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/3 "2018-04-26T16:53:10Z")

</div>

Read [this](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and specifically the "Also be patient" part.

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 26, 2018, 4:55pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/4 "2018-04-26T16:55:00Z")

</div>

Which space is filled? Does not look obvious to me.

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [April 27, 2018, 7:36am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/5 "2018-04-27T07:36:59Z")

</div>

/app directory is getting filled. The log generation is huge and elastic search stopped after it reach to 85%. Today it reached to 79%.

```auto
[fuseadmin@a0110pcsgmon03 ~]$ df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/rhel_vg-root_lv 33G 1.9G 29G 6% /
devtmpfs 32G 0 32G 0% /dev
tmpfs 32G 0 32G 0% /dev/shm
tmpfs 32G 449M 31G 2% /run
tmpfs 32G 0 32G 0% /sys/fs/cgroup
/dev/sda1 488M 147M 306M 33% /boot
/dev/mapper/rhel_vg-home_lv 30G 160M 28G 1% /home
/dev/mapper/rhel_vg-var_lv 20G 1.3G 18G 7% /var
/dev/mapper/app_vg-app_lv 1.5T 1.1T 300G 79% /app
tmpfs 6.3G 0 6.3G 0% /run/user/1009
[fuseadmin@a0110pcsgmon03 ~]$ df -kh | grep -i /app
/dev/mapper/app_vg-app_lv 1.5T 1.1T 300G 79% /app
[fuseadmin@a0110pcsgmon03 ~]$

```

Let me know if way to delete some old logs or way to analysis more deathly.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 27, 2018, 8:46am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/6 "2018-04-27T08:46:53Z")

</div>

Please format your code. I did it twice for you but please do it to make it easier to read. The preview window will help a lot. You can use markdown style.

So `/app` is where you put your elasticsearch data, right?

What gives:

```auto
GET _cat/nodes?v
GET _cat/indices?v

```

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [April 27, 2018, 11:19am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/7 "2018-04-27T11:19:33Z")

</div>

yes. location is .

[fuseadmin@a0110pcsgmon03 elasticsearch-5.5.0]$ pwd  
/app/elasticsearch-5.5.0  
[fuseadmin@a0110pcsgmon03 elasticsearch-5.5.0]$  
[fuseadmin@a0110pcsgmon03 elasticsearch-5.5.0]$ ll  
total 256  
drwxr-xr-x. 2 fuseadmin fueadmin 4096 Apr 26 18:59 bin  
drwxr-xr-x. 3 fuseadmin fueadmin 4096 Apr 26 15:40 config  
drwxr-xr-x. 3 fuseadmin fueadmin 4096 Sep 23 2017 data  
drwxr-xr-x. 2 fuseadmin fueadmin 4096 Jul 1 2017 lib  
-rw-r--r--. 1 fuseadmin fueadmin 11358 Jul 1 2017 LICENSE.txt  
drwxr-xr-x. 2 fuseadmin fueadmin 16384 Apr 27 00:14 logs  
drwxr-xr-x. 13 fuseadmin fueadmin 4096 Jul 1 2017 modules  
-rw-------. 1 fuseadmin fueadmin 0 Apr 26 18:58 nohup.out  
-rw-r--r--. 1 fuseadmin fueadmin 194187 Jul 1 2017 NOTICE.txt  
drwxr-xr-x. 2 fuseadmin fueadmin 4096 Jul 1 2017 plugins  
-rw-r--r--. 1 fuseadmin fueadmin 9548 Jul 1 2017 README.textile

You mentioned -"Please format your code. I did it twice for you but please do it to make it easier to read. "

Please let me know where and also let me know  
GET \_cat/nodes?v  
GET \_cat/indices?v

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 27, 2018, 11:34am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/8 "2018-04-27T11:34:15Z")

</div>

I did that for your first posts.

Please format your code, logs or configuration files using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.  
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.  
Please update your post.

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 27, 2018, 11:34am UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/9 "2018-04-27T11:34:59Z")

</div>

Please provide what is the output of the cat API I asked for.

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [April 29, 2018, 8:27pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/10 "2018-04-29T20:27:29Z")

</div>

ip heap.percent ram.percent cpu load\_1m load\_5m load\_15m node.role master name  
127.0.0.1 41 93 13 0.76 0.61 0.57 mdi \* DiGiNode..

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [April 29, 2018, 8:39pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/11 "2018-04-29T20:39:53Z")

</div>

```auto
yellow open fuselog-2018.04.07 qkUqviZmTe6F4MhG_7Cxjg 5 1 3168101 0 16.4gb 16.4gb
yellow open fuselog-2018.04.21 gF-izQo4RT-QNBxcO9HkFQ 5 1 15487399 0 58gb 58gb
yellow open fuselog-2018.04.09 ySwGtyXCR7aO4yFQ4DEQDg 5 1 3209478 0 16.7gb 16.7gb
yellow open fuselog-2018.04.18 muSCHehkRR62rugWr1TEEA 5 1 8280513 0 42.6gb 42.6gb
yellow open fuselog-2018.02.04 PJLLWc4RQZWxpOYlTQqBFw 5 1 40367 0 123mb 123mb
yellow open fuselog-2017.10.23 gZ5vPqyPSC-OuZXoRRM5jw 5 1 68 0 286kb 286kb
yellow open fuselog-2018.03.07 BKYQfiCOQ9yEgGJoQxYUiA 5 1 85264 0 245.6mb 245.6mb
yellow open fuselog-2018.03.17 AaPjQr2YRhCuTTFaJ4vjPA 5 1 70310 0 168.3mb 168.3mb
yellow open fuselog-2017.11.06 UBLfwR4HTTyL03bHEdeFtA 5 1 13489 0 52.5mb 52.5mb
yellow open fuselog-2018.02.07 Fvt2Z_IJSVmrJ0s0M1eseA 5 1 76747 0 185.9mb 185.9mb
yellow open fuselog-2018.01.10 56G8VssWQwKt5FxpIg6QYQ 5 1 67867 0 232mb 232mb
yellow open fuselog-2018.04.25 2VzrP0m6RuCwtsld55bIOw 5 1 20058747 0 73.8gb 73.8gb
yellow open fuselog-2017.10.04 C03WwyPWTTKgwXyVSJEZhw 5 1 465 0 889.4kb 889.4kb
yellow open fuselog-2017.11.07 B0ZDrowPTDWCm_KNMwiALQ 5 1 12790 0 50mb 50mb
yellow open fuselog-2017.11.26 bQNnutBqRbGWfTHfkEHdaw 5 1 11520 0 44.7mb 44.7mb
yellow open fuselog-2018.02.23 2nvpCBWwT9OVfGzYOnyhXg 5 1 75475 0 184.4mb 184.4mb
yellow open fuselog-2018.01.15 tsjljB8OTOmCUpKZgGcO8w 5 1 77502 0 260.2mb 260.2mb
yellow open fuselog-2018.01.02 udJnR03XQOy6TBceMteKLA 5 1 19767 0 75.1mb 75.1mb
yellow open fuselog-2017.11.09 MJVD92JpRSyao3ZdUnQQfA 5 1 11273 0 43.4mb 43.4mb
yellow open fuselog-2018.02.02 zljRRzlPQhqhNUrX69g2HA 5 1 37740 0 112.4mb 112.4mb
yellow open fuselog-2018.01.01 opuxEctbQqGBB4LyKJs33A 5 1 22198 0 84.9mb 84.9mb
yellow open fuselog-2018.02.01 YFRLdxdUQfKfWEOVdjeBBw 5 1 37182 0 117.7mb 117.7mb
yellow open fuselog-2018.01.13 e_WXVw-lSVGzezySrJO8YQ 5 1 140233 0 449.8mb 449.8mb
yellow open fuselog-2018.04.14 O4Cs4zLXRvqgwzfeKLfy9w 5 1 5916013 0 31.1gb 31.1gb
yellow open fuselog-2017.11.23 o25xcTXcQV6biL04Gez1Rg 5 1 110738 0 126.6mb 126.6mb
yellow open fuselog-2018.04.20 NGGZkWc8QiC7EzmN4TFUPg 5 1 15995158 0 57.9gb 57.9gb
yellow open fuselog-2017.11.19 Rrzz3VEbTH-gFqC4nhdKTA 5 1 11206 0 44.5mb 44.5mb
yellow open fuselog-2017.10.30 4nPdUpfKSySCb6bO-RZyzg 5 1 13549 0 52.6mb 52.6mb
yellow open .kibana fU0pVNukQG6VJi8dFBCD-g 1 1 32 2 82.2kb 82.2kb
yellow open fuselog-2017.11.04 6qQ0l9jjTN2qpJ8o4WlCQg 5 1 14564 0 56.1mb 56.1mb
yellow open fuselog-2018.04.22 ctBUjBEOSkOchcNUYxHT3Q 5 1 15670490 0 58.5gb 58.5gb

```

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [April 29, 2018, 8:51pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/12 "2018-04-29T20:51:49Z")

</div>

Please format your code, logs or configuration files using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) and not the citation button. It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.  
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.  
Please update your post.

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [May 3, 2018, 12:22pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/13 "2018-05-03T12:22:23Z")

</div>

```
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
127.0.0.1 10 91 40 4.80 4.56 4.49 mdi * DiGiNode
    indent preformatted text by 4 spaces
```

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [May 3, 2018, 12:23pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/14 "2018-05-03T12:23:27Z")

</div>

```
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open fuselog-2018.05.03 Ivek1mr5S9q1KiRDu-2NUg 5 1 12269800 0 49.5gb 49.5gb
yellow open .kibana MtbGHB3TSz6KivnuRarZQQ 1 1 31 0 108.1kb 108.1kb`Preformatted text`
```

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [May 4, 2018, 4:30pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/15 "2018-05-04T16:30:27Z")

</div>

Any update

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [May 4, 2018, 5:31pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/16 "2018-05-04T17:31:49Z")

</div>

Are you sure you still need the old indices from 2017 ?

Anyway, apparently since the end of April you are injecting a lot of data.

What is the sum of all space that the \_cat API is showing? Is it something like 1100 gb?  
If not, run may be:

```auto
du -s /app

```

To see which dir is taking most of the space.

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [May 7, 2018, 12:57pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/17 "2018-05-07T12:57:06Z")

</div>

[fuseadmin@a0110pcsgmon04 ~]$ du -sh --time /app  
du: cannot read directory â/app/lost+foundâ: Permission denied  
513G 2018-05-07 20:56 /app  
[fuseadmin@a0110pcsgmon04 ~]$

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [May 7, 2018, 1:20pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/18 "2018-05-07T13:20:00Z")

</div>

Can you find which dir under app is taking all space? Did you sum as I asked the size given by the cat API?

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [May 7, 2018, 1:26pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/19 "2018-05-07T13:26:57Z")

</div>

GET /\_cat/allocation?v

shards disk.indices disk.used disk.avail disk.total disk.percent host ip node  
26 504gb 580.5gb 895.8gb 1.4tb 39 localhost 127.0.0.1 DiGiNode  
26

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size  
yellow open fuselog-2018.05.05 pGGh2mwASGmHET\_YKSgldA 5 1 28867041 0 103.4gb 103.4gb  
yellow open fuselog-2018.05.07 UQvSXBKlSjeJM\_bngM01wQ 5 1 35098404 0 124.6gb 124.6gb  
yellow open data AlnpmM8QSeiFciLtJTkDCA 5 1 0 0 810b 810b  
yellow open fuselog-2018.05.06 GqrrGFuvT56cWfCdEjcO8g 5 1 30833617 0 112.2gb 112.2gb  
yellow open .kibana MtbGHB3TSz6KivnuRarZQQ 1 1 32 2 80.6kb 80.6kb  
yellow open fuselog-2018.05.04 Cq1Cluf8SEK6F2wv3NFNGw 5 1 45942884 0 164.8gb 164.8gb

[fuseadmin@a0110pcsgmon04 elasticsearch-5.5.0]$ du -sh --time \*  
1.1M 2018-05-07 21:25 bin  
24K 2018-05-03 19:16 config  
520G 2018-05-07 21:25 data  
22M 2017-07-01 07:20 lib  
12K 2017-07-01 07:14 LICENSE.txt  
824K 2018-05-07 21:25 logs  
14M 2017-07-01 07:20 modules  
192K 2017-07-01 07:20 NOTICE.txt  
4.0K 2017-07-01 07:20 plugins  
12K 2017-07-01 07:14 README.textile  
[fuseadmin@a0110pcsgmon04 elasticsearch-5.5.0]$

[fuseadmin@a0110pcsgmon04 nodes]$ du -sh --time \*  
520G 2018-05-07 21:26 0  
12K 2018-05-07 20:51 1  
[fuseadmin@a0110pcsgmon04 nodes]$

[fuseadmin@a0110pcsgmon04 nodes]$ cd 0 && du -sh --time \*  
521G 2018-05-07 21:26 indices  
0 2017-09-13 20:11 node.lock  
12K 2018-05-07 20:35 \_state  
[fuseadmin@a0110pcsgmon04 0]$

---

<div class="post-metadata">

### Author: ![Tamal\_Kundu](https://avatars.discourse-cdn.com/v4/letter/t/439d5e/32.png) [@Tamal\_Kundu](https://discuss.elastic.co/u/Tamal_Kundu)
#### Post date: [May 7, 2018, 1:27pm UTC](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647/20 "2018-05-07T13:27:36Z")

</div>

Please let me know if your require more any more output

[Next page](https://discuss.elastic.co/t/space-is-getting-filled-need-some-information-how-to-clear-re-claim-the-space/129647.md?page=2)
