SNMPTRAP input behind F5 load balancer - Service Monitor generating lots of errors in log

Elastic Stack Logstash
1

Hello,

We have a cluster of logstash instances behind an F5 load balancer. Part of the F5 implementation includes a health check that is sent every couple of seconds to make sure the UDP port on the logstash is still up. If the health check fails then the instances is declared dead and the F5 will not send traffic to it anymore.

The F5 can send a string over TCP/UDP and can do an API call.

Today the UDP test to Logstash is not a real SNMP trap thus an error message is created saying that the trap could not be parsed.

Received data:
"default send string"
Error handling trap: 100
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/snmp-1.3.1/lib/snmp/ber.rb:192:in decode_sequence' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/snmp-1.3.1/lib/snmp/pdu.rb:50:in decode'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/snmp-1.3.1/lib/snmp/manager.rb:710:in block in process_traps' org/jruby/RubyKernel.java:1411:in loop'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/snmp-1.3.1/lib/snmp/manager.rb:707:in process_traps' /usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-snmptrap-3.0.6/lib/logstash/inputs/snmptrap/patches/trap_listener.rb:17:in process_traps'
/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/snmp-1.3.1/lib/snmp/manager.rb:634:in `block in initialize'

Is there a way to not process these Health Check messages in the pipeline? Maybe a way to drop these logs when they come from a specific source?

Anyone else have experience with F5 in front of logstash for Traps?

Thanks

2

No, but that said.... If you are using a clustered F5 pair then (depending on your config) you may be able to tell from the source IP whether it is a health check. The health check will come from the base IP of the F5, not the cluster address. If that does not help I would lean towards configuring a separate input for the F5 to poll. Instead of hitting the snmp input, have it check an http input, then discard all events from that input.

3

Thank you for the response.

I was trying to figure how to do that in the INPUT config of logstash since we don't get to the filter portion of the config since the trap is malformed. Can you suggest a way to do this?

input {
snmptrap {
id => "snmp-trap-input"
port => "1062"
codec => json
community => ["public","SNMP-trap"]
}
# END INPUT
}

I agree that I may get an HTTP endpoint created, but could I use the same port? The network engineer I am dealing with says it does not make sense using a different port.

4

I understand why the network engineer would think that, but you want to test if logstash is responsive. Whilst possible, it is very unlikely that one input would stop responding but another would not.

You can configure the text sent as part of the health check. Can you make it a valid SNMP trap?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.