# Filebeat events are not distributed equally to the logstash pods

**URL:** <https://discuss.elastic.co/t/filebeat-events-are-not-distributed-equally-to-the-logstash-pods/376153>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [March 20, 2025, 3:32am UTC](https://discuss.elastic.co/t/filebeat-events-are-not-distributed-equally-to-the-logstash-pods/376153 "2025-03-20T03:32:53Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nageswarrao\_Bandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nageswarrao_bandi/32/142149_2.png) [@Nageswarrao\_Bandi](https://discuss.elastic.co/u/Nageswarrao_Bandi)\
**Post date:** [March 20, 2025, 3:32am UTC](https://discuss.elastic.co/t/filebeat-events-are-not-distributed-equally-to-the-logstash-pods/376153/1 "2025-03-20T03:32:53Z")

</div>

I am using Filebeat, Logstash, Elastic search, Kibana in my Kubernetes cluster to monitor all my application logs.

Let me explain the issue and share my configurations.

1. Filebeat collecting the logs and sending the events to the logstash pods, But I cloud observes load is not evenly distributed to the pods, tried with k8s service Cluster IP and Headless service.

2. Due to uneven the load my logstash queues are failing up and not processing as fast as input events from the filebeat.

Filebeat config:

`output.logstash: hosts: ['logstash-headless.logging.svc:5044'] loadbalance: true`

Logstash Config:

`queue.type: persisted path.queue: /usr/share/logstash/queue queue.max_bytes: 8gb queue.page_capacity: 512mb pipeline.batch.size: 9000 pipeline.batch.delay: 7`

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 20, 2025, 4:04am UTC](https://discuss.elastic.co/t/filebeat-events-are-not-distributed-equally-to-the-logstash-pods/376153/2 "2025-03-20T04:04:08Z")

</div>

Hello and welcome,

In your case the configuration needs to be a little diferent.

The `loadbalance` option in the output should be used when beats are responsible for the load balancing, which means, when you have **multiple** hosts configured in the output, in your case you have only one hosts and the load balancing is done externally, not by beats.

Also, the connection from beats to Logstash is sticky, this can lead to uneven distribution when the destination host is a load balancer, which is your case, to avoid this you need to specify a `ttl`.

So, your output would need to be something like this:

```auto
output.logstash:
  hosts: ["logstash-headless.logging.svc:5044"]
  loadbalance: false
  pipelining: 0
  ttl: 5m

```

The `pipelining` is a setting related to async batches in Logstash, but since this does not work when `ttl` is used, you need to set it to `0`.

You can change the value of the `ttl` if you want.

The documentation for these settings can be found [here](https://www.elastic.co/guide/en/beats/filebeat/8.17/logstash-output.html#loadbalance).

---

<div class="post-metadata">

**Author:** ![Nageswarrao\_Bandi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nageswarrao_bandi/32/142149_2.png) [@Nageswarrao\_Bandi](https://discuss.elastic.co/u/Nageswarrao_Bandi)\
**Post date:** [March 20, 2025, 7:55am UTC](https://discuss.elastic.co/t/filebeat-events-are-not-distributed-equally-to-the-logstash-pods/376153/3 "2025-03-20T07:55:36Z")

</div>

Thank you! @leandrojmp For quick response. Will follow your suggestions.