Thanks for the reply Ry,
my question was more related to how to guarantee a smooth transition if I make the upgrade at some point in time say x.
I would like to avoid the condition where messages indexed before x have logs as type and messages indexed after x have type doc.
This is how I was thinking to modify the logstash configuration file, given that x is some point in time on 6th February and new indexes are created at night when @timestamp hour is 01:00
filter{
...
if [@metadata][type] == "system_logs"
{
mutate{
# enable timestamp comparison adding a new field
add_field => { "str_dt_new" => "2020-02-07T01:00:00.000Z"}
}
date {
match => ["str_dt_new", "YYYY-MM-dd'T'HH:mm:ss.SSSZ"]
target => "constant_date"
}
}
if [@timestamp] <= [constant_date] {
mutate {
# for consistency with messages sent before changing config file, in the
# same day (i.e. same ES index)
replace => { "type" => "logs" }
}
}
else {
mutate {
# required from LS v6.x
replace => { "type" => "doc" }
}
}
}
Does it make sense to you?