Elasticsearch appears to be unreachable or down

Elastic Stack Elasticsearch
1

We suddenly started seeing a huge increase ( from single digits to hundreds) in the error
"Attempted to send a bulk request to elasticsearch' but Elasticsearch appears to be unreachable or down!}"
This is causing significant drop (80%) in the amount of data making it to elastic cluster (8 data nodes and 3 master).
we are running 7.4 logstash and elastic
logstash pulls from kafka and pushes to elastic
about 50 logstash instances
any help will be appreciated

2

Check your heap usage and GC rates, when those go bad, it can happen quick.

3

thanks for the response. What are ideal values for GC count and duration rate? JVM heap is between max of 22.1 and min of 4.7. Mostly staying around 10 GB. GC duration is ~33ms with young at 0.5 avg

4

Compare to before the problem, watch for percent of time in GC.

What is IO wait?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.