Filebeat slower than logstash default "file" input?


So far I have been using logstash's default "file" input plugin. I am giving filebeat a try with logstash.

Filebeat gives me a significant performance boost when it comes to reading large logfiles, this is great.

However, I noticed something else:
When appending a single log line manually to logstash through the filebeat input plugin and a simple stdout output without any additional processing in the middle, my payload is taking a lot longer to be processed (I'm talking >5seconds delay complared to logstash input plugin).

My filebeat config is bare minimum. Is there anything I can do to improve this ?


maybe sharing your configuration files and more setup details (OS, filebeat/logsthas version, filesystem) might be a good start.

The 5 second delay sounds like you've enabled multiline support.


  • Linux Mint 18
  • filebeat 1.3.1 (amd64)
  • Logstash 2.4.0
  • partition where logs are written is ext4 on a SSD drive

Config files:

$ cat filebeat.yml

        - "/tmp/TEST.*.messages.log"

    hosts: [""]

$ cat logstash.conf

input {
  beats {
    port => 5044

output {
    stdout { codec => rubydebug }

hope that helps

If a file is not updated very frequently, filebeat backs off to not constantly poll the file. I assume before you added the line manually, the file wasn't updated for some time before. You can configure how much filebeat backs off with the config options here:

gotcha. good to know, thanks a lot !

This topic was automatically closed after 21 days. New replies are no longer allowed.