How to filter Errors only from logs? using logstash

Elastic Stack Logstash
1

Hi, I am using logstash in our project, How to filter debug and info log from logs ? and saved separate file?

i am using below configuration:

input {
file {
path => "/home/kunchala/TESTING/mywork/trap_testing/consul.log"
start_position => "beginning"
}
}
filter {

    grok{
  match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{LOGLEVEL:LEVEL} %{GREEDYDATA:whateverElseYoureFiltering}" }
    }

}
output {
if [level] == "DEBUG" {
file{
path => "/home/kunchala/TESTING/mywork/trap_testing/DEBUG.log"
}
}
else if [level] == "INFO" {
file{
path => "/home/kunchala/TESTING/mywork/trap_testing/INFO.log"
}
}

 else {
     file{
        path  => "/home/kunchala/TESTING/mywork/trap_testing/MO.log"
       }
      }
 stdout {    codec =>     rubydebug }
}
2

And what is the problem you have with the result of that?

3

my concern is all debug information i want to save in DEBUG.log and all info log information need to be save in INFO.log file. but that one is not happening. all the debug , info ,etc logging information stored in MO.log file . could you please help me on this anything i did wrong

4

Please provide an example log line.

5

{"message":"2018-07-25_16:57:48.39018 consul 2018/07/25 11:57:48 [DEBUG] agent: Service 'consul' in sync\n","@version":"1","crs":"consul","mms":" 2018/07/25 11:57:48 [DEBUG] agent: Service 'consul' in sync\n","host":"205.26.198.10","ts":"2018-07-25_16:57:48.39018","@timestamp":"2018-07-25T16:57:48.389Z"}

6

Is that what a line of the log file looks like? If so, it is not even close to matching your grok filter. It has a timestamp that I would not expect to match %{SYSLOGTIMESTAMP} followed by a hostname and a second timestamp.

Build your grok filter one field at a time and make sure the first part matches before adding additional fields.

And for that format dissect is probably better than grok.

7

i am new to logstash, can you please send me the filter for that.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.