Logstash udp input queue_size

Maybe the total throughput of your logstash configuration is slower than the input rate? Logstash is as fast as the slowest part of the configuration. Maybe your filters are slow? Maybe your outputs are slow?

You can find out the slowest/busiest part of logstash with:

top -p <pid> -H

This will show you something like:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 8725 logstash  20   0 9242m 792m  14m S 22.9  0.6 328:18.40 >output
 8382 logstash  20   0 9242m 792m  14m S  4.7  0.6  58:40.69 <redis
 8416 logstash  20   0 9242m 792m  14m S  3.0  0.6  38:34.98 |worker
 8418 logstash  20   0 9242m 792m  14m S  3.0  0.6  38:26.75 |worker

In my case the output plugins are the slowest/busiest component. I guess you will find a component using 100% CPU.

If you have a bottleneck try increasing the number of threads (using the workers plugin parameter or the filterworkers commandline parameter), so your logstash instance can process more events per second.

If that does not improve your throughput you really should think about horizontal scaling: adding more logstash instances. Using UDP will not really work anymore in such a setup. You may want to investigate a queueing middleware like Redis.

Hope this helps.