Given the following configuration,
filebeat.spool_size: 786432 filebeat.idle_timeout: 30s output.elasticsearch: enabled: true hosts: ["elastic-node1:9200", "elastic-node2:9200", "elastic-node3:9200"] ... worker: 64 bulk_max_size: 4096 flush_interval: 30s compression_level: 3
It seems to me that
output.elasticsearch.flush_interval settings have very similar roles - allow to wait for the spool / bulk to fill up for some time, albeit not on the same level.
Does it make sense to enable both of them, and especially the
filebeat.idle_timeout is already set?