# Elasticsearch curator delete disk space

**URL:** <https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450>\
**Category:** Elasticsearch\
**Tags:** curator\
**Created:** [June 26, 2019, 1:38am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450 "2019-06-26T01:38:41Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ohollx](https://avatars.discourse-cdn.com/v4/letter/o/ce7236/32.png) [@ohollx](https://discuss.elastic.co/u/ohollx)\
**Post date:** [June 26, 2019, 1:38am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/1 "2019-06-26T01:38:41Z")

</div>

Hi.

I'm trying to delete an index automatically using an elastic curator.  
I think the index is deleted when setting as below, but I do not understand exactly how it works.

actions:  
One:  
action: delete\_indices  
options:  
ignore\_empty\_list: True  
timeout\_override: 300  
continue\_if\_exception: False  
disable\_action: False  
filters:  
- filtertype: pattern  
kind: prefix  
value: wazuh-  
- filtertype: space  
disk\_space: 0.3  
use\_age: True  
source: creation\_date

When the individual index reaches the capacity set in disk\_space, the old index is deleted from the oldest index. If the index list is deleted, it also includes logs that do not meet the capacity set in disk\_space.  
And I do not know how much of the entire index is deleted.

What I would like to do is to delete 10% of the total log from the old log when the index total capacity reaches the capacity set in disk\_space.

I would appreciate your help.  
thank you.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 26, 2019, 1:56am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/2 "2019-06-26T01:56:23Z")

</div>

> [@ohollx](#):
>
> ```auto
> filters:
> - filtertype: pattern
> kind: prefix
> value: wazuh-
> - filtertype: space
> disk_space: 0.3
> use_age: True
> source: creation_date
> 
> ```

What these filters do:

1. Select only indices beginning with `wazuh-`
2. Sum the space of all indices selected by any and all previous filters, in this case, the filter in step 1. Delete all indices in excess of the specified amount of `disk_space` in gigabytes, starting with the oldest (determined by `use_age: true`).

What you're doing is deleting all `wazuh-` indices using in excess of 300 Mbytes. You can see the math used by running Curator with `--dry-run` and setting `loglevel: DEBUG` in the client settings yaml file.

---

<div class="post-metadata">

**Author:** ![ohollx](https://avatars.discourse-cdn.com/v4/letter/o/ce7236/32.png) [@ohollx](https://discuss.elastic.co/u/ohollx)\
**Post date:** [June 26, 2019, 2:24am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/3 "2019-06-26T02:24:16Z")

</div>

At run time, the following result is displayed. If you check the store.size of the delete file, there is no index of 300mb.

# curator --configcurator-config.yml delete\_indices\_space.yml --dry-run

2019-06-26 11:12:58,982 INFO Preparing Action ID: 1, "delete\_indices"  
2019-06-26 11:12:58,991 INFO Trying Action ID: 1, "delete\_indices": No description given  
2019-06-26 11:12:59,081 INFO DRY-RUN MODE. No changes will be made.  
2019-06-26 11:12:59,081 INFO (CLOSED) indices may be shown that may not be acted on by action "delete\_indices".  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.03 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.04 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.05 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.06 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.07 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.08 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.09 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.10 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.11 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.12 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.13 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.14 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-alerts-3.x-2019.05.15 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.03 with arguments: {}  
2019-06-26 11:12:59,082 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.05 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.06 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.07 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.08 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.09 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.10 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.11 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.12 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.13 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.14 with arguments: {}  
2019-06-26 11:12:59,083 INFO DRY-RUN: delete\_indices: wazuh-monitoring-3.x-2019.05.15 with arguments: {}  
2019-06-26 11:12:59,084 INFO Action ID: 1, "delete\_indices" completed.  
2019-06-26 11:12:59,084 INFO Job completed.

If you run disk\_space with 200mb, more indexes will be deleted than 300mb. I do not understand this part.

The store.size in each individual index file is up to 18mb, which is less than 200 or 300mb. I wonder if the index is selected in some criteria.  
index Is not the capacity reference store.size?

I would appreciate your help.  
thank you.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 26, 2019, 3:11am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/4 "2019-06-26T03:11:53Z")

</div>

You're showing which indices were set to be deleted, but not the log lines which show how the space math is calculated.

> [@ohollx](#):
>
> At run time, the following result is displayed. If you check the store.size of the delete file, there is no index of 300mb.

As stated, Curator selects indices when the total sum of all indices is **in excess** of `disk_space` gigabytes, beginning with the newest (most recent) indices, as it will delete older indices **in excess** of the amount specified by `disk_space`. Curator does not look for a single index of that size. It's a sum.

---

<div class="post-metadata">

**Author:** ![ohollx](https://avatars.discourse-cdn.com/v4/letter/o/ce7236/32.png) [@ohollx](https://discuss.elastic.co/u/ohollx)\
**Post date:** [June 26, 2019, 6:35am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/5 "2019-06-26T06:35:11Z")

</div>

Thanks to the friendly explanation, I understood.

But there is one thing that I do not understand.  
If disk\_size is set to 300mb, it will be deleted from the oldest index except 300mb index.

However, curator's index size calculation is strange.  
This is indices store size as confirmed by GET \_stats below.  
"store": {  
"size\_in\_bytes": 4447439230  
},

I'm wondering if the size of the indices is above,  
If disk\_size of curator is set to 3G, deletion list should be generated, but there is no deletion list.

I do not know if the calculation method is wrong or there is something wrong with me.

I would appreciate your help.

thank you.

```
action: delete_indices
options:
  ignore_empty_list: True
  timeout_override: 300
  continue_if_exception: False
  disable_action: False
filters:
- filtertype: pattern
  kind: prefix
  value: wazuh-
- filtertype: space
  disk_space: 3
  use_age: True
  source: creation_date
```

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [June 26, 2019, 12:27pm UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/6 "2019-06-26T12:27:36Z")

</div>

Please paste the section of the DEBUG output which shows how the sizes are summed up and we can go over it together. It should be pretty clear how it sums the space consumed by the indices, starting from the most recent, so that all older indices above your 3G threshold are selected for the associated action (in this case, `delete_indices`).

---

<div class="post-metadata">

**Author:** ![ohollx](https://avatars.discourse-cdn.com/v4/letter/o/ce7236/32.png) [@ohollx](https://discuss.elastic.co/u/ohollx)\
**Post date:** [June 27, 2019, 12:48am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/7 "2019-06-27T00:48:33Z")

</div>

I confirmed my mistake while confirming what you told me.  
The size search method is invalid.  
Thanks to me, I solved a problem that was difficult for me.  
It has helped a lot.  
thank you.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 25, 2019, 12:48am UTC](https://discuss.elastic.co/t/elasticsearch-curator-delete-disk-space/187450/8 "2019-07-25T00:48:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.