bump.
added curator for delete old indices and delete by disk space - work.
remain : rollover index every 5gb or 1day
my logstash config
input {
tcp {
port => 5556
}
udp {
port => 5566
}
}filter {
csv {
separator => ","
columns => [
"os","reporting_computer","client_time" ,"full_server_time" ,"process_id" ,"process_name" ,
"process_path" ,"protocol" ,"status" ,"source_port" ,"destination_port" ,"direction" ,"cast_type",
"scramble_state" ,"source_ip" ,"destination_ip" ,"sequance_number" ,"sub_sequance_number" ,"user_name" ,
"mog_counter" ,"destination_path" ,"reason" ,"dll_path" ,"dll_name" ,"chain_array"
]
}
mutate {convert => ["process_id","integer"]}
mutate {convert => ["source_port","integer"]}
mutate {convert => ["destination_port","integer"]}
mutate {convert => ["sequance_number","integer"]}
mutate {convert => ["mog_counter","integer"]}
}
output {
elasticsearch {
hosts => "http://localhost:9200"
index => "logs-%{+YYYY.MM.dd}"
template => "C:\etc\logstash-config\index_template.json"
template_overwrite => "true"
}
}
curator settings:
> actions:
> 1:
> action: delete_indices
> description: >-
> Delete index's older than X days or when reach disk space 200gb
> options:
> ignore_empty_list: True
> filters:
> - filtertype: pattern
> kind: prefix
> value: logs-
> - filtertype: age
> source: name
> direction: older
> timestring: '%Y.%m.%d'
> unit: days
> unit_count: 60
> - filtertype: space
> disk_space: 400
> use_age: True
> source: field_stats
> field: '@timestamp'
> stats_result: max_value
>
> actions:
> 1:
> action: rollover
> description: >-
> Rollover the index every 1gb.
> options:
> name: logs_write
> conditions:
> max_size: 5g
> max_age: 1d
something wrong with my rollover curator - he cant find logs_Write name aliases
i added to my elastic template this alias.
there is an option via logstash to roll every 5gb or every 1day ? (already got every 1 day)