I'm ingesting Netflow data using Filebeat 7.14.0 and writing the output into Elasticsearch. I'm trying to tweak the various configuration settings in order to improve the performance (currently, it seems like filebeat/ES cannot handle the incoming load.)
I see from the filebeat monitoring logs that
libbeat.pipeline.queue.max_events: 4096. I wanted to increase this to see if the performance would improve.
filebeat.yml contains only the
filebeat.config.modules section, while the
filebeat.inputs are effectively not enabled.
netflow.yml is as follows.
- module: netflow log: enabled: true var: netflow_host: 0.0.0.0 netflow.port: 1234 queue_size: 8192
I'm not sure if setting
queue_size: 8192 in
netflow.yml is helping because I don't see this anywhere on the filebeat monitoring log, and
libbeat.pipeline.queue.max_events remains at
4096 despite multiple restarts of filebeat.
Where do I configure