Filebeat.idle_timeout VS output.elasticsearch.flush_interval

Elastic Stack Beats
1

Hi,

Given the following configuration,

filebeat.spool_size: 786432
filebeat.idle_timeout: 30s

output.elasticsearch:
  enabled: true
  hosts: ["elastic-node1:9200", "elastic-node2:9200", "elastic-node3:9200"]
  ...
  worker: 64
  bulk_max_size: 4096
  flush_interval: 30s
  compression_level: 3

It seems to me that filebeat.idle_timeout and output.elasticsearch.flush_interval settings have very similar roles - allow to wait for the spool / bulk to fill up for some time, albeit not on the same level.

Does it make sense to enable both of them, and especially the output.elasticsearch.flush_interval if filebeat.idle_timeout is already set?

Thanks,
MG

2

The filebeat spooler circumvents the additional batching in the outputs pipeline. Only filebeat.idle_timeout should be active here.

1 Like
3

Note, with 6.0.0 beta1 both options are removed.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.