Fatal level logs are not processing in Logstash

Elastic Stack Logstash
1

Hi All,

We have an architecture where the log-flow process starts from Beats -> AWS MSK (Kafka) -> Logstash (Self hosted)-> Elasticsearch -> Kibana.

But we are facing a challenge where the FATAL level logs are coming till Kafka and not getting filtered from the Logstash when we are using the below filter :slight_smile:
filter {

    json {
            source => "message"

}

fingerprint {
   source => [ "fingerprint", "log" ]
   target => "[@metadata][fingerprint]"
   method => "SHA1"
   concatenate_sources => true
}

}

But when we remove the filter "json {
source => "message"
}", we are able to get the logs along with the message field.

Could anyone please suggest on how to get the FATAL logs without making any change in the filter part?

Regards,
Nalin

2

Hi,

FATAL log are in JSON format?

Regards