ES is not coming up after changing JAVA_HOME

Prabaharan_Kathiresa · July 5, 2021, 12:49pm

Hi Team,

We are running ES 6.8.5 in unix machine and its running on 1.8. We decided to promote java 1.8 to java11 oracle jdk. Can someone advise me what should i do now ?

Steps we did

Stopped ES 6.8.5 from Unix Machine
Uninstalled 1.8 from the unix machine and installed Java11 oracle jdk
Changed JAVA_HOME in the elasticsearch file to point java11 (not in elasticsearch.yml. i can see two files, one is elasticsearch & another is elasticsearch.yml)
I started the ES server and ES server is not coming up and following is the ES log


[2021-07-05T11:54:01,143][INFO ][o.e.e.NodeEnvironment    ] [FyXSBu_] using [1] data paths, mounts [[/app (/dev/mapper/vgapp-app)]], net usable_space [598.2gb], net total_space [600.7gb], types [xfs]
[2021-07-05T11:54:01,151][INFO ][o.e.e.NodeEnvironment    ] [FyXSBu_] heap size [989.8mb], compressed ordinary object pointers [true]
[2021-07-05T11:54:01,152][INFO ][o.e.n.Node               ] [FyXSBu_] node name derived from node ID [FyXSBu_2R-ya-xFrksnDbw]; set [node.name] to override
[2021-07-05T11:54:01,153][INFO ][o.e.n.Node               ] [FyXSBu_] version[6.8.5], pid[4878], build[default/tar/78990e9/2019-11-13T20:04:24.100411Z], OS[Linux/3.10.0-1160.25.1.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/11.0.10.0.1/11.0.10.0.1+1-LTS-2]
[2021-07-05T11:54:01,153][INFO ][o.e.n.Node               ] [FyXSBu_] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-8901506701007211350, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.path.home=/app/elasticsearch-6.8.5, -Des.path.conf=/app/elasticsearch-6.8.5/config, -Des.distribution.flavor=default, -Des.distribution.type=tar]
[2021-07-05T11:54:02,355][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [aggs-matrix-stats]
[2021-07-05T11:54:02,355][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [analysis-common]
[2021-07-05T11:54:02,355][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [ingest-common]
[2021-07-05T11:54:02,355][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [ingest-geoip]
[2021-07-05T11:54:02,355][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [ingest-user-agent]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [lang-expression]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [lang-mustache]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [lang-painless]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [mapper-extras]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [parent-join]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [percolator]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [rank-eval]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [reindex]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [repository-url]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [transport-netty4]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [tribe]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-ccr]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-core]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-deprecation]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-graph]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-ilm]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-logstash]
[2021-07-05T11:54:02,356][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-ml]
[2021-07-05T11:54:02,357][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-monitoring]
[2021-07-05T11:54:02,357][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-rollup]
[2021-07-05T11:54:02,357][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-security]
[2021-07-05T11:54:02,357][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-sql]
[2021-07-05T11:54:02,357][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-upgrade]
[2021-07-05T11:54:02,357][INFO ][o.e.p.PluginsService     ] [FyXSBu_] loaded module [x-pack-watcher]
[2021-07-05T11:54:02,357][INFO ][o.e.p.PluginsService     ] [FyXSBu_] no plugins loaded
[2021-07-05T11:54:04,995][INFO ][o.e.x.s.a.s.FileRolesStore] [FyXSBu_] parsed [0] roles from file [/app/elasticsearch-6.8.5/config/roles.yml]
[2021-07-05T11:54:05,470][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [FyXSBu_] [controller/5031] [Main.cc@109] controller (64 bit): Version 6.8.5 (Build b07c2d9433ee6d) Copyright (c) 2019 Elasticsearch BV
[2021-07-05T11:54:05,830][DEBUG][o.e.a.ActionModule       ] [FyXSBu_] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2021-07-05T11:54:05,997][INFO ][o.e.d.DiscoveryModule    ] [FyXSBu_] using discovery type [zen] and host providers [settings]
[2021-07-05T11:54:06,634][INFO ][o.e.n.Node               ] [FyXSBu_] initialized
[2021-07-05T11:54:06,635][INFO ][o.e.n.Node               ] [FyXSBu_] starting ...
[2021-07-05T11:54:06,795][INFO ][o.e.t.TransportService   ] [FyXSBu_] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2021-07-05T11:54:09,886][INFO ][o.e.c.s.MasterService    ] [FyXSBu_] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {FyXSBu_}{FyXSBu_2R-ya-xFrksnDbw}{4mivzUQ1RwS2JqzXWa6plg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=33566654464, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2021-07-05T11:54:09,890][INFO ][o.e.c.s.ClusterApplierService] [FyXSBu_] new_master {FyXSBu_}{FyXSBu_2R-ya-xFrksnDbw}{4mivzUQ1RwS2JqzXWa6plg}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=33566654464, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {FyXSBu_}{xxxxxxxx}{xxxx}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=33566654464, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2021-07-05T11:54:09,945][INFO ][o.e.h.n.Netty4HttpServerTransport] [FyXSBu_] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2021-07-05T11:54:09,947][INFO ][o.e.n.Node               ] [FyXSBu_] started
[2021-07-05T11:54:10,093][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [FyXSBu_] Failed to clear cache for realms [[]]
[2021-07-05T11:54:10,119][INFO ][o.e.l.LicenseService     ] [FyXSBu_] license [xxxxxxxx] mode [basic] - valid
[2021-07-05T11:54:10,126][INFO ][o.e.g.GatewayService     ] [FyXSBu_] recovered [0] indices into cluster_state

warkolm · July 5, 2021, 10:34pm

That means Elasticsearch has indeed started.

Prabaharan_Kathiresa · July 6, 2021, 1:07pm

Initially I thought of ES services are started but I could not connect to ES neither through elastic head plugin nor cerebros plugin.

Prabaharan_Kathiresa · July 7, 2021, 12:05pm

Hi warkolm

We are using OracleJdk11, through ansible below ES command is failing,

elasticsearch-keystore list

ERROR is could not find

JAVA_HOME

We set the Java_home for all users in unix. so everyone java & javac commands are pointing to oraclejdk11.

is that anything to do with oraclejdk11 ?

system · August 4, 2021, 12:05pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.