Logstash - 1 event ends up 22 times in elasticsearch

Hi everyone,

got a real strange issue right now, haproxy is sending 1 event but logstash send it 22 times to elasticsearch.

i did only 1 change in the config (tested before in a test environment, without any problems)

normal config

if [logstash] == "haproxy_log" {
        grok {  id => match_haproxy_pattern
                match => {"message" => "%{HAPROXYHTTP}"}
                match => {"message" => "%{HAPROXYTCP}"}

changed config

if [logstash] == "haproxy_log" {
        grok {  id => match_haproxy_pattern
               patterns_dir => ["/etc/logstash/patterns.d/"]
               match => {"message" => [ "%{HAPROXYHTTP}", "%{HAPROXYTCP}", "%{HAPROXYHTTP2}", "%{HAPROXYTCP2}" ]}

the reason for the change was simple, i need to remove from some frontends the client_ip:port in the logs.

so i just changed the patterns and put both in it.

The question would be now:

did i miss something with the grok filter?
does the change put a big pressure on the performance? and that maybe could explain it?

also at the time when i did the restart, logs were going up to 70,000 instead of 2,000 per 5minute interval.

thanks in advance for any information

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.