Cannot connect to the Elasticsearch cluster currently configured for Kibana after the upgrade to 6.4


(Rahma) #1

Hi Elastic Team,
I did an upgrade of the Elastic cluster from the 6.3.2 to the 6.4, and then I cannot connect to the Elasticsearch cluster currently configured for Kibana.
When I check the kibana logs , message error is below " Port 5601 is already in use. Another instance of kibana may be running"
lsod -i :5601
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
node 1223 kibana 11u IPv4 27239 0t0 TCP hostname.domain

and Elasticsearch service failed, error is the Elastic logs "ElasticsearchUncaughtExceptionHandler] [SIEM] fatal error in thread [Thread-6], exiting
java.lang.OutOfMemoryError: Java heap space"

OS: CentOS7
Elasticsearch version : 6.4
Kibana version : 6.4
java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-b10)
OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)

Thanks


(Jon Budzenski) #2

For kibana, can you share a little about your upgrade process? If the server's already running it may be as simple as restarting it so it can reload the new files.

For es, I'm guessing when the elasticsearch cluster is coming back online, shards are being moved causing a spike in memory usage. Does the server have available memory and are java.options configured to use less memory (should be ~half) of the available memory?


(Rahma) #3

For Kibana, it was an sample install of rpm package with rpm -e of the 6.3.2 version. Restart is done , kibana go online for few minutes and then stop.
At that moment , Elasticsearch service fails .
For ES: cluster go back and down , it is not stable.
java -XshowSettings:all
VM settings:
Max. Heap Size (Estimated): 13.92G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM
please advice.
Thanks


(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.