Elasticsearch index and curator

Elastic Stack Elasticsearch
1

Hi all!
I got a newbie question for you :slight_smile:
Right now, I have configured my logstash destination to my elastic only node like this:
ex:

if [type] == "syslog"  {
        elasticsearch {
        hosts => localhost
        index => "logstash-%{+YYYY.MM.dd}" }
        }

So if I understand correctly, this will create a new index everyday, am I right?
Also, I have a curator script running that should clean the data older than 30 days:

---
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 45 days (based on index name), for logstash-
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.
    options:
      ignore_empty_list: True
      timeout_override:
      continue_if_exception: False
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: '^(logstash-).*$'
      exclude:
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 7
      exclude:

But I am not sure that I am doing this the right way since if I check the status of my elastic node, some shards (using cerebro) some shards do not seems to be assigned...

So should I create my index like "name-of-the-index" instead of "name-of-the-index-%{+YYYY.MM.dd}"?

Is my curator script will work anyway? Do my shards will be assigned this time?

I know,,. plenty of question.. :slight_smile:
Thanks for your comments!

2

Unassigned shards is a different matter. You mention an elastic node as a single entity. If you only have a single node, then you will see unassigned shards by default, as there will be no other node on which to allocate replica shards.

This is overkill. The prefix filtertype already takes whatever value you provide and translates it into a regex like ^VALUE.*$ So you can do this:

filters:
  - filtertype: pattern
    kind: prefix
    value: logstash-

Also note that empty values will just take the default, so you don't need the exclude: line.

Other than that, it looks good.

3

Thanks!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.