Need to create a field for multiple type exceptions


(Narmatha) #1

Hi

I have configured logstash configuration to create an index in ES for all the logs present in three node cluster. Please find the below conf.

input {
  file {
    path => "D:\Development_Avecto\logstash-2.4.1\bin\dev_exceptions.txt"
    start_position => "beginning"
    sincedb_path => "NUL"
    codec => multiline {
            pattern => "^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]"
            negate => true
            what => "previous"
        }
  }
}
filter {
    grok {
      match => [ "message", "(?m)^\[%{TIMESTAMP_ISO8601:TIMESTAMP}\]\[%{LOGLEVEL:LEVEL}%{SPACE}\]\[%{DATA:ERRORTYPE}%{SPACE}\]%{SPACE}(?<ERRORMESSAGE>(.|\r|\n)*)"]
    }
# DEBUG Logs
if "grokked" not in [tags] and "DEBUG" == [LEVEL] {
grok { match => [ "ERRORMESSAGE", "(?m)^\[%{DATA:SERVERNAME}\]" ]
add_tag => [ "Debug Logs", "grokked" ]
tag_on_failure => [ ]
}
}
# INFO Logs
if "grokked" not in [tags] and "INFO" == [LEVEL] {
grok { match => [ "ERRORMESSAGE", "(?m)^\[%{DATA:SERVERNAME}\]" ]
add_tag => [ "Info Logs", "grokked" ]
tag_on_failure => [ ]
}
}
# WARN Logs
if "grokked" not in [tags] and "WARN" == [LEVEL] {
grok { match => [ "ERRORMESSAGE", "(?m)^\[%{DATA:SERVERNAME}\]" ]
add_tag => [ "Warn Logs", "grokked" ]
tag_on_failure => [ ]
}
}
if "Exception" in [ERRORMESSAGE] {
    mutate { add_field => { "Exception" => "Exception" } 
                }
}
date {
    match => [ "TIMESTAMP" , "yyyy-MM-dd HH:mm:ss,SSS" ]
    target => "TIMESTAMP"
  }
mutate {
        remove_field => ["message","@version","path","host","tags","type" ]
      }
}
output {
elasticsearch { hosts => ["localhost:9200"]
     index => "logstash-devlogs"
}
stdout { codec => rubydebug }
}

As per the conf file, a new field called "Exception"(specified in If condition) has created for the mentioned index.

And i need one more raw field should be created for the "Exception" field with respect to errors(for Eg: if it is a mapper parse exception or search parse exception or Invalid exception or Timeout exception)
Kindly post your comments ASAP. very urgent.


(Magnus B├Ąck) #2

You should be able to do pretty much the same thing that you did for messages containing "Exception", or...?

Giving a concrete example of what you want to accomplish would make it easier to help.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.