We're using the following curator script to expire data older than 7 days.
actions:
1:
action: delete_indices
description: >-
Delete indices older than 7 days (based on index name), for logstash-
prefixed indices. Ignore the error if the filter does not result in an
actionable list of indices (ignore_empty_list) and exit cleanly.
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: logstash-
exclude:
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 7
exclude:
We have data from several different users, we would like some users to have a different index retention than others. What would we the best way to achieve this?
Right now the names of our indices start with logstash-Y.m.d
One way I thought it would be possible is to have several indices depending on the retention, for example for 14 day retention logstash14d-%Y.%m.%d for 30 days logstash30d-%Y.%m.%d
The downside to this is that we would have to check before ingesting every log event what the users retention is.