Increasing queue size

Elastic Stack Beats
1

Hi,

I'm ingesting Netflow data using Filebeat 7.14.0 and writing the output into Elasticsearch. I'm trying to tweak the various configuration settings in order to improve the performance (currently, it seems like filebeat/ES cannot handle the incoming load.)

I see from the filebeat monitoring logs that libbeat.pipeline.queue.max_events: 4096. I wanted to increase this to see if the performance would improve.

My filebeat.yml contains only the filebeat.config.modules section, while the filebeat.inputs are effectively not enabled.

My netflow.yml is as follows.

- module: netflow
  log:
    enabled: true
    var:
      netflow_host: 0.0.0.0
      netflow.port: 1234
      queue_size: 8192

I'm not sure if setting queue_size: 8192 in netflow.yml is helping because I don't see this anywhere on the filebeat monitoring log, and libbeat.pipeline.queue.max_events remains at 4096 despite multiple restarts of filebeat.

Where do I configure libbeat.pipeline.queue.max_events?

Thank you.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.