Logstash 6.4.1 & 6.5.0 High CPU usage

monitoring

(Jan) #1

Hi All,

I upgraded Elastic Stack from:

  • At first 6.3.0 --> 6.4.1 (whole stack)
  • Then from 6.4.1 --> 6.4.2 (only Logstash)
  • then from 6.4.2 --> 6.5.0 (only Logstash)

After first upgrade I have notice high CPU usage change on logstash nodes comparing to version 6.3.0.
In consequence Logstash stopped consume logs from Kafka.

Then decided to move to version 6.4.2 all goes back to normal (still higher comparing to 6.3.0 but in tolerance).

Then decided to update to new release - CPU usage goes up again!

What could be possible reason for high CPU usage? No error logs or warn logs which could point to the problem.

For comparison please see below CPU usage on node that runs 6.3.0 version:

It seems to be similar problem to this one: Logstash 6.2.2 high CPU usage

Will appreciate your help,
Thanks,
Jan


(Jan) #2

Hi an todays update.

Logstash 6.5.0 is down, not consuming any logs from kafka & keeps 100% cpu utilization

For comparison on boxes with logstash 6.3.0 everything looks perfect:

From logs:

[2018-11-22T10:50:28,199][INFO ][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] [Consumer clientId=xyz.kafka.uat.elastic.consumer-0, groupId=logstash] Attempt to heartbeat failed since group is rebalancing

Will appreciate your help,
Thanks,
Jan