Logstash filter conditions not working correctly!

Hello Dears,

Logstash.conf:

I will bold the conditions in the filter that did not work correctly. I made sure that using filter query in Elasticsearch UI it retrieved messages containing these error conditions, but Logstash configuration did not retrieve anything.

filter {
    
     if '/opt/app/tomcat/logs/payment-default.log' in [log][file][path]{ 
     **if 'java.net.ConnectException: Connection refused (Connection refused)' in [message]** {
     mutate {
       add_field => {"document_id" => "%{id}"}
       add_field => {"body"        => "%{message}"}
     }
    }
    **else if 'Connection Refused' in [message]** {
     mutate {
       add_field => {"document_id" => "%{id}"}
       add_field => {"body"        => "%{message}"}
      }
     }
    else if 'Load has thrown exception' in [message] {
     mutate {
       add_field => {"document_id" => "%{id}"}
       add_field => {"body"        => "%{message}"}
     }
     }
     else if 'HTTP/1.1 403 Forbidden' in [message] {
     mutate {
       add_field => {"document_id" => "%{id}"}
       add_field => {"body"        => "%{message}"}     }
     }
     else { drop{} }
     }
     else { drop{} }
    
}

Any help please!

Any Help!!!!!

You need to share an example of a message that should match but it is not matching so people can try to replicate your issue.

Without it there is no way to see what the issue could be as conditionals are pretty simple, if they are not matching probably your test condition is not correct.

[message]:
Logs: ERROR o.s.a.r.l.SimpleMessageListenerContainer:1223 - Failed to check/redeclare auto-delete queue(s).
org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:62)

Can't replicate, it works for me without any problem.

sample file:

Logs: ERROR o.s.a.r.l.SimpleMessageListenerContainer:1223 - Failed to check/redeclare auto-delete queue(s).
org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:62)
Logs: sample text to make multiline work

pipeline used to test

input { 
    stdin {
        codec => multiline {
            pattern => "^Logs:"
            negate => true
            what => "previous"
        }
    }
}
filter {
     if 'java.net.ConnectException: Connection refused (Connection refused)' in [message] {
        mutate {
          add_field => {"document_id" => "123"}
          add_field => {"body"        => "%{message}"}
        }
     }
}
output {
    stdout {}
}

output:

{
           "body" => "Logs: ERROR o.s.a.r.l.SimpleMessageListenerContainer:1223 - Failed to check/redeclare auto-delete queue(s).\norg.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)\nat org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:62)",
           "host" => "elk",
           "tags" => [
        [0] "multiline"
    ],
        "message" => "Logs: ERROR o.s.a.r.l.SimpleMessageListenerContainer:1223 - Failed to check/redeclare auto-delete queue(s).\norg.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)\nat org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:62)",
    "document_id" => "123",
     "@timestamp" => 2022-01-13T14:42:45.341Z,
       "@version" => "1"
}

Since you have an conditional, inside anothe conditional, is your first conditional working?

yes it is working fine, I did not use "multiline" in my input.

As I said, I can't replicate, the conditional works fine for me, the multiline I've used was just for the example as the example message you shared is a multiline message.

Can you share the json of the entire document that should have matched, but didn't match? You can get it in Discover in Kibana, share the full json for the document using the Preformatted text (</>).

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.