# Curator age filter not matcing all indices

**URL:** <https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083>\
**Category:** Elasticsearch\
**Tags:** curator\
**Created:** [June 24, 2019, 8:42am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083 "2019-06-24T08:42:16Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [June 24, 2019, 8:42am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/1 "2019-06-24T08:42:16Z")

</div>

Hi,

We are creating daily indices and have a need of automatically closing them when they get older than 3 months. I have the following in my Curator action file:

```
14:
    action: close
    description: >-
      Close indices older than 3 months that are allocated to the archive-node
    options:
      delete_aliases: False
      disable_action: False
      ignore_empty_list: False
    filters:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y-%m-%d'
      unit: months
      unit_count: 3
    - filtertype: allocated
      key: box_type
      value: archive

```

This is the name of an example index that is not being closed:

**logstash-azurestack-2019-02-28**

```
{
  "logstash-azurestack-2019-02-28" : {
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "require" : {
              "box_type" : "archive"
            }
          }
        },
        "refresh_interval" : "30s",
        "number_of_shards" : "1",
        "provided_name" : "logstash-azurestack-2019-02-28",
        "creation_date" : "1551312020299",
        "number_of_replicas" : "0",
        "uuid" : "Kg0gYT7dTSylxlyupV1r3w",
        "version" : {
          "created" : "6030299"
        }
      }
    }
  }
}

```

How come the index isn't getting closed?

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [June 24, 2019, 9:13am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/2 "2019-06-24T09:13:46Z")

</div>

Hi @victor.nilsson,

What is the o/p of curator logs showing. could you please post it for analysis?

Regards,  
Harsh Bajaj

---

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [June 24, 2019, 9:27am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/3 "2019-06-24T09:27:27Z")

</div>

Sure.

> 2019-06-23 20:18:42,719 INFO Preparing Action ID: 14, "close"  
> 2019-06-23 20:18:42,722 INFO Trying Action ID: 14, "close": Close indices older than 3 months that are allocated to the archive-node  
> 2019-06-23 20:19:13,755 INFO Skipping action "close" due to empty list: \<class 'curator.exceptions.NoIndices'\>  
> 2019-06-23 20:19:13,755 INFO Action ID: 14, "close" completed.  
> 2019-06-23 20:19:13,755 INFO Job completed.

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [June 24, 2019, 9:32am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/4 "2019-06-24T09:32:11Z")

</div>

Hi @victor.nilsson,

Could you please enable the Debug mode in curator.yml file and paste the debug logs instead of INFO?

It will help me to understand where is the problem.

Regards,  
Harsh Bajaj

---

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [June 24, 2019, 9:52am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/5 "2019-06-24T09:52:59Z")

</div>

Sure thing. The output is pretty damn large, is there a way to share it to you?

EDIT:

```
2019-06-24 11:50:55,287 DEBUG curator.indexlist __not_actionable:39 Index logstash-iazurestack-2019-02-28 is not actionable, removing from list.
2019-06-24 11:50:55,287 DEBUG curator.indexlist __excludify:58 Removed from actionable list: logstash-iazurestack-2019-02-28: Routing (mis)match: index.routing.allocation.require.box_type=archive.

```

I can see that it mismatches on the routing allocation awareness. However, the index is on the archive-node as mentioned in the OP.

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [June 24, 2019, 9:55am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/6 "2019-06-24T09:55:28Z")

</div>

Could you please try with dry run without using below section and check it is showing in actionable list or not.

```
filtertype: allocated
key: box_type
value: archive

```

if its working then please try below configuration for archive. You need to edit "options" section and add below parameters.

> ```
> options:
> key: box_type
> value: archive
> allocation_type: require
> wait_for_completion: True
> filters:
> 
> ```

Regards  
Harsh Bajaj

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [June 24, 2019, 10:17am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/7 "2019-06-24T10:17:13Z")

</div>

@victor.nilsson,

> [@victor.nilsson](#):
>
> We are creating daily indices and have a need of automatically closing them when they get older than 3 months.

You have not define any pattern in your action file. So you should define the pattern and your action file will be look like below:

```auto
actions:
  1:
    action: close
    description: >-
      Close indices older than 30 days (based on index name), for logstash-
      prefixed indices.
    options:
      delete_aliases: False
      timeout_override:
      continue_if_exception: False
      disable_action: True
    filters:
    - filtertype: pattern
      kind: prefix
      value: logstash-azurestack-
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 90
      exclude:

```

Please try above action file. If you face any issue let me know.

Thanks.

---

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [June 24, 2019, 11:04am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/8 "2019-06-24T11:04:27Z")

</div>

> [@harshbajaj16](#):
>
> Could you please try with dry run without using below section and check it is showing in actionable list or not.
> 
> ```
> filtertype: allocated
> key: box_type
> value: archive
> 
> ```

This did not work. I got the same errors as before.

---

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [June 24, 2019, 11:07am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/9 "2019-06-24T11:07:23Z")

</div>

Hi, Thanks but you're missing the crucial feature that we need. We only want to close indices that are allocated to our archive node.

```
- filtertype: allocated
      key: box_type
      value: archive

```

I tried your configfile with the allocated filtertype added and still got the same error.

---

<div class="post-metadata">

**Author:** ![victor.nilsson](https://avatars.discourse-cdn.com/v4/letter/v/eb8c5e/32.png) [@victor.nilsson](https://discuss.elastic.co/u/victor.nilsson)\
**Post date:** [June 24, 2019, 11:18am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/10 "2019-06-24T11:18:24Z")

</div>

I solved it by adding "exclude: false" to the allocated filter.

> - filtertype: allocated  
> key: box\_type  
> value: archive  
> exclude: false

Thanks all for the help

---

<div class="post-metadata">

**Author:** ![harshbajaj16](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/harshbajaj16/32/44970_2.png) [@harshbajaj16](https://discuss.elastic.co/u/harshbajaj16)\
**Post date:** [June 24, 2019, 11:22am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/11 "2019-06-24T11:22:16Z")

</div>

Yes,i shared the some properties which you can add/remove as per your requirement.

Great if it works!!

---

<div class="post-metadata">

**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\
**Post date:** [June 24, 2019, 11:31am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/12 "2019-06-24T11:31:00Z")

</div>

@victor.nilsson,

> [@victor.nilsson](#):
>
> I solved it by adding "exclude: false" to the allocated filter.

Great..yes by default exclude is true in filtertype allocated. I didn't check it earlier.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 22, 2019, 11:31am UTC](https://discuss.elastic.co/t/curator-age-filter-not-matcing-all-indices/187083/13 "2019-07-22T11:31:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.