# Logstash: Resource consumption of multiple Logstash servers are not even

**URL:** <https://discuss.elastic.co/t/logstash-resource-consumption-of-multiple-logstash-servers-are-not-even/260406>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [January 7, 2021, 4:55am UTC](https://discuss.elastic.co/t/logstash-resource-consumption-of-multiple-logstash-servers-are-not-even/260406 "2021-01-07T04:55:35Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Hung\_Phan\_Huy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hung_phan_huy/32/80736_2.png) [@Hung\_Phan\_Huy](https://discuss.elastic.co/u/Hung_Phan_Huy)\
**Post date:** [January 7, 2021, 4:55am UTC](https://discuss.elastic.co/t/logstash-resource-consumption-of-multiple-logstash-servers-are-not-even/260406/1 "2021-01-07T04:55:36Z")

</div>

Hi,

I'm running my Logstash services on Docker Swarm with 4 replicas. They subscribe to the same Kafka topic and output the data to Elasticsearch.

When I inspect the resource consumption of those 4 containers, I found that the memory is even, but the CPU is not. 2 containers consume roughly 50% CPU but the other 2 never exceed 2%.

Am I having an issue with the other 2 containers with low CPU consumption?

Some useful logs in Logstash containers with low CPU consumption:

```
[2021-01-07T05:02:56,787][INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] Discovered group coordinator 10.128.151.164:9092 (id: 2147482646 rack: null),
[2021-01-07T05:02:56,790][INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] (Re-)joining group,
[2021-01-07T05:02:59,785][INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] Successfully joined group with generation 923,
[2021-01-07T05:02:59,787][INFO][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] Adding newly assigned partitions: ,
[2021-01-07T05:03:08,789][INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] Attempt to heartbeat failed since group is rebalancing,
[2021-01-07T05:03:08,790][INFO][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] Revoke previously assigned partitions ,
[2021-01-07T05:03:08,790][INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] (Re-)joining group,
[2021-01-07T05:02:56,786][INFO][org.apache.kafka.clients.Metadata][main][75103990350a1b4ea8e9c7e115f84685510dc8689748dfad19b2d52677e6a771] [Consumer clientId=logstash-0, groupId=logstash] Cluster ID: BIbPq3azQnSIjx2QURxtBA

```

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![Hung\_Phan\_Huy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hung_phan_huy/32/80736_2.png) [@Hung\_Phan\_Huy](https://discuss.elastic.co/u/Hung_Phan_Huy)\
**Post date:** [January 7, 2021, 6:55am UTC](https://discuss.elastic.co/t/logstash-resource-consumption-of-multiple-logstash-servers-are-not-even/260406/2 "2021-01-07T06:55:19Z")

</div>

The reason is that I only have 2 partitions for my Kafka topics so only 2 Logstash instances can subscribe to those 2 partitions. The other 2 are useless.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 4, 2021, 6:55am UTC](https://discuss.elastic.co/t/logstash-resource-consumption-of-multiple-logstash-servers-are-not-even/260406/3 "2021-02-04T06:55:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.