Curator: ignore_empty_list & warn_if_no_indices

Elastic Stack Elasticsearch
1

I am using Curator to add & remove indexes from an alias (curator_action.yml below). I have ignore_empty_list set to True. The indexes in question relate to an application server in our test environment that are used infrequently. Indexes are created with a %Y.%m suffix.

If either the 'add' OR 'remove' section return no indexes I need to set warn_if_no_indices to True to have Curator run successfully.
If both the 'add' AND 'remove' sections return no indexes I need to set warn_if_no_indices to False to have Curator run successfully.

I realize that I can create 2 seperate actions, one containing the 'add' and the second action containing the remove. (Your reply to this ticket: Curator Alias Add not working if Remove list is empty)

Is there any way to have Curator run successfully given the 2 scenarios above?

It appears to me that creating 2 seperate actions is the only option when working with indexes where either or both of the add & remove sections can be empty as there does not appear to be a combination of settings that will cater for all scenarios (either or both of the add & remove sections can be empty).

I'm also not sure (anymore - I thought I knew what ignore means) what the object of the ignore_empty_list option is ....

curator_action.yml

action: alias
description: Alias indices
options:
  disable_action: True
  ignore_empty_list: True
  name: MyIndex-historical
  warn_if_no_indices: True
add:
  filters:
  - filtertype: pattern
    kind: prefix
    value: idx_MyIndex-
    exclude:
  - filtertype: period
    source: name
    range_from: -3
    range_to: -2
    timestring: '%Y.%m'
    unit: months
    week_starts_on: monday
remove:
  filters:
  - filtertype: pattern
    kind: prefix
    value: idx_MyIndex-
  - filtertype: period
    source: name
    range_from: -7
    range_to: -4
    timestring: '%Y.%m'
    unit: months
    week_starts_on: monday
2

Sorry you're having a hard time. You seem to be in between a rock and a hard place with this. What you're trying to do, however, is not clean. Elasticsearch will return an error if you attempt to make an update alias API call without a body in it. The body contains the add and remove sections, and if there are no indices in both the add and remove sections, that will result in an exception. Accordingly, Curator cannot make an API call to Elasticsearch without something in it, so Curator raises its own exception, rather than send nothing to Elasticsearch and catch that exception. There is light at the end of the tunnel for your use case, however. Let me explain what's going on in the code and why, before explaining that part.

The alias action is special. The ignore_empty_list option only exists to catch a particular kind of exception, which is that the entire action failed with a NoIndices exception, with the test being in the CLI code, rather than in the action. For only the alias action, setting warn_if_no_indices: True effectively causes Curator to not raise the NoIndices exception. As a result, for the alias action, ignore_empty_list is useless if you've set warn_if_no_indices: True.

Here's what's going on under the hood.

  1. Because warn_if_no_indices is set to True, the warn_if_no_indices checks here and here mean that Curator gets the go-ahead to attempt the alias update API call, even when there are no indices selected.

  2. Curator will attempt do_action, which will generate the body on demand, here and here.

  3. When both the add and the remove filters result in empty lists, there is nothing in self.actions, which results in an ActionError exception being raised.

This is what Curator should be doing. It's telling you that you hit an unrecoverable exception—that it cannot make an API call with no adds or removes in it. However, you want Curator to ignore this exception, because you, the operator, know that this might happen. The way to do so is to set continue_if_exception: True in your options, as you want to bypass this legitimate exception.

3

Aaron,

This to say thanks for a prompt response with a detailed & knowledgeable answer.

Cheers - Thanks again!
Sean

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.