Logstash UDP input only processing ~1000 events/sec

Elastic Stack Logstash
1

I have an 8core 32GB Windows VM running LS 2.3.2 ES 2.3.3 Kibana 4.5.1 and I max out the CPUs with 4 UDP workers and process ~1,000 UDP netflow events/sec. This seems really low compared to what I have seen other rates. It is the basic input and output to ES - no filters.. Any ideas why I can't process any more with an 8core? Changing udp queue size also didn't make a difference..

input {
udp {
port => 9995
workers => 4
codec => netflow {
definitions => "c:\elk\logstash\vendor\bundle\jruby\1.9\gems\logstash-codec-netflow-2.0.5\lib\logstash\codecs\netflow\netflow.yaml"
versions => [9]
}
}
}

output {
elasticsearch {
index => "logstash_netflow-%{+YYYY.MM.dd}"
hosts => ["localhost:9200"]
}
}

2

Have you measured what kind of indexing performance you can get from your ES cluster, i.e. have you ruled out ES as the bottleneck? 1000 events/s seems low, but it's not unreasonably low for a one-node ES cluster.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.