Curator and the close command

Elastic Stack Elasticsearch
1

How do I use curator's "close" command feature with age of indices? The documentation page only shows this:

$ curator close --help
Usage: curator close [OPTIONS] COMMAND [ARGS]...

Close indices

Options:
--help Show this message and exit.

Commands:
indices Index selection.

I need to close indices based on either space or age, but the documentation does not show how it's done. Please help. Thank you!

2

The "Getting Started" section helps you to navigate the nested command structure, particularly The Command-Line Interface and Examples

3

I've looked on that page already and it does not state how to close indices days/minutes/hours old. It shows this:

curator --host 10.0.0.2 close indices --exclude do-not-touch --exclude logstash-2015.03

4

So I tried the following command, and I think I'm getting closer:

curator close indices --older-than 1 --time-unit hours

2015-11-09 16:30:12,125 ERROR Parameters --older-than and --newer-than
require the --timestring parameter

Can you clarify what it's asking for as far as the --timestring parameter?
My logstash index format is "logstash-2015.11.09"

5

Ah, I see. You're looking for specifics, and the documentation does not do this.

The examples and docs are non-specific because every command that uses the indices sub-command uses the same Index Selection parameters, including the close command:

! Important
This command requires the indices subcommand for index selection.

The links in the documentation are to help explain the usage of the nested command/subcommand structure. The same "indices" arguments that work in one example will work for all.

6

See the timestring documentation. It has links to the python strftime formatting examples that will help you.

That would result in a timestring of %Y.%m.%d

7

So how would that command look? Could you check mine below and tell me what is wrong?

curator close indices --older-than 1 %Y.%m.%d --time-unit hours
2015-11-09 17:21:51,806 ERROR Parameters --older-than and --newer-than require the --timestring parameter

8

Now i'm getting a different error:

curator close indices --older-than 1 --time-unit hours 1 --timestring %Y.%m.%d
Usage: curator close indices [OPTIONS]

Error: Got unexpected extra argument (1)

9

For one, you're using hours but your timestring is daily. You can't use hours unless %H appears in your timestring.

The unexpected argument comes between hours and --timestring. You don't need that 1 there. That's what's causing the error.

10

Got it. Thanks Aaron. My final command worked:

curator close indices --older-than 1 --time-unit days --timestring %Y.%m.%d

2015-11-09 18:04:55,857 INFO Job starting: close indices
2015-11-09 18:04:55,915 WARNING No indices matched provided args: {'regex': None, 'index': (), 'suffix': None, 'newer_than': None, 'closed_only': False, 'prefix': None, 'time_unit': 'days', 'timestring': u'%Y.%m.%d', 'exclude': (), 'older_than': 1, 'all_indices': False}
No indices matched provided args: {'regex': None, 'index': (), 'suffix': None, 'newer_than': None, 'closed_only': False, 'prefix': None, 'time_unit': 'days', 'timestring': u'%Y.%m.%d', 'exclude': (), 'older_than': 1, 'all_indices': False}

11

Hi Aaron-

One last question...

When indices are closed, what exactly happens to them? Does it move to a different directory? How are they distinguishable apart from open indices?

12

Indices do not move. They are only marked as closed and unavailable for search.

There are only 2 things Elasticsearch can do with closed indices: open them, and delete them. Even deletes are a brief "open" followed by an immediate delete.

Elasticsearch cannot calculate the size of closed indices. Elasticsearch does not keep tabs on how much disk-space closed indices consume. If you close indices, you will need to rely on operating-system-level disk usage monitoring.