I started using the community catalog entry of elasticsearch under Rancher .
It is a cluster of 2 data nodes, 1 master node, 1 client node.
I started seeing a disconnect of Elasticsearch changing to status green to red with Request timeout.
work around followed:
By checking the forum,
- I changed my timeout parameter under EL to 60000ms.
Did not help that much.
- Under monitoring, I saw more usage of JVM and logs showed gc overhead.
So changed Java Xmx and Xms parameter to 8g from default of 2g.
Little bit of help it did.
and I saw a different error now whenever a POST request is sent to EL
9/14/2017 11:27:37 AM log [16:27:37.114] [error][admin][elasticsearch] Request error, retrying
9/14/2017 11:27:37 AMPOST http://elasticsearch:9200/.kibana/config/_search => socket hang up
- set the following parameters in elasticsearch yml
All of these workaround did not help that much. Still the response is very slow in Kibana. and EL plugin changes to red.
I just have only one index running. 6.4 GB data. Current JVM memory usage:
Any other possible causes for this kind of error?
Any workaround to solve this issue is much appreciated.