How to Rotate Output log which is processed by logstash

Elastic Stack Logstash
1

HI,

I am trying to parse a parent log file and try to store the logs into separate output files based on keyword.

Sample code i have used for creating a logstash.conf file can be found below

input {
file {
path => ["Path to input file "]

 type => "log"
start_position => "beginning"

}
}

filter {

grok {
match => { "message" => "%{TIME:timestamp}%{SPACE}%{LOGLEVEL:level}%{SPACE}[(?[^]]+)]%{SPACE}((?[^)]+))%{SPACE}%{WORD:tenant}%{SPACE}%{DATA:message}" }

}
}

output {

if "KeyWord" in [variable_name]{
file {
path => ["/%{variable_name}.log"]

}
}
}

The above code creates output files per based on the key word that is present in the input files.

I am looking to rotate the out files generated by this logstatsh.conf files.

Request you to help me on this.

2

path => ["/%{variable_name}.log"]

I guess you're paraphrasing here, but the first directory component may not contain field references.

I am looking to rotate the out files generated by this logstatsh.conf files.

Do you want the files rotated based on size or date or something else?

If you rotate the files externally (with e.g. logrotate) Logstash will unfortunately pick that up and you need to restart Logstash after the files have been rotated (see issue #10).

If it's okay to get one file per day (or month or hour or minute ...) you can include e.g. %{+YYYY.MM.dd}in the filename pattern to have Logstash include the date from the @timestamp field in the actual filename.