How to retrieve n number of previous lines in log file using logstash configuration?

Elastic Stack Logstash
1

I am creating an application where I need to put previous and after n number of lines in elasticsearch with current log. This is my current logstash configuraion.
What changes I need to make so that I can retrieve last n number of lines(Let's say 5) in my output?

input{ file{ path => "D:\ELK_Info\TestLogs_Updated.log"
#start_position => beginning
    ignore_older => 0
    sincedb_path => "NUL"
        codec => multiline {
                               pattern => "^%{TIMESTAMP_ISO8601}"
                               negate => true
                               what => "previous"
        } } }

filter{
    grok{
       match => {
      "message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{DATA:line_number} %{DATA:log_level} %{DATA:log_type} %{NOTSPACE:space} %{GREEDYDATA:stackTrace}"    }   }   mutate { remove_field => [ "tags", "space", "line_number"]} }

output { file{ path => "D:\ELK_Info\logstashOutput.log" }
2

Can you give an example of the file TestLogs_Updated.log?
What do you mean previous and after n number of lines?
How the multiline codec works if it searches the pattern and takes as many lines until the pattern appears again. Do you have such a pattern in your log?

3

@redX
Consider that current log starts from {TIMESTAMP_ISO8601} but previous lines will not have any particular structure.
Can we just retrieve n number of previous lines from this current log?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.