Upgrade to ES 6.0, ElasticSearch wont start

Elastic Stack Elasticsearch
1

I have upgraded from ElasticSearch 5.6.2 to 6.0.0 on Centos 7 and am getting the following error when trying to start the service:
Nov 24 16:18:37 desktopoli systemd: Starting Elasticsearch...
Nov 24 16:18:37 desktopoli systemd: Failed at step EXEC spawning /usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec: No such file or directory
Nov 24 16:18:37 desktopoli systemd: elasticsearch.service: control process exited, code=exited status=203
Nov 24 16:18:37 desktopoli systemd: Failed to start Elasticsearch.
Nov 24 16:18:37 desktopoli systemd: Unit elasticsearch.service entered failed state.
Nov 24 16:18:37 desktopoli systemd: elasticsearch.service failed.

Any idea on how to fix?

2

Hi, can you provide Elasticsearch logs as well ?

Something like :
tail -100f /var/log/elasticsearch/your_cluster_name.log

3

It seems that when path.data is not configured(commented out) in elasticsearch.yml, in 6.0 ElasticSearch will try to configure /usr/share/elasticsearch/logs/ and log there.
Anyway, this is where I am at now:

[2017-11-24T16:54:41,111][INFO ][o.e.n.Node ] [] initializing ...
[2017-11-24T16:54:41,184][INFO ][o.e.e.NodeEnvironment ] [ana-9ag] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [29.6gb], net total_space [43.9gb], types [rootfs]
[2017-11-24T16:54:41,184][INFO ][o.e.e.NodeEnvironment ] [ana-9ag] heap size [990.7mb], compressed ordinary object pointers [true]
[2017-11-24T16:54:41,185][INFO ][o.e.n.Node ] node name [ana-9ag] derived from node ID [ana-9agrRay_xty0esFTMg]; set [node.name] to override
[2017-11-24T16:54:41,185][INFO ][o.e.n.Node ] version[6.0.0], pid[116238], build[8f0685b/2017-11-10T18:41:22.859Z], OS[Linux/3.10.0-514.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_102/25.102-b14]
[2017-11-24T16:54:41,185][INFO ][o.e.n.Node ] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch]
[2017-11-24T16:54:41,835][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [aggs-matrix-stats]
[2017-11-24T16:54:41,835][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [analysis-common]
[2017-11-24T16:54:41,835][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [ingest-common]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [lang-expression]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [lang-mustache]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [lang-painless]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [parent-join]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [percolator]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [reindex]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [repository-url]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [transport-netty4]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] loaded module [tribe]
[2017-11-24T16:54:41,836][INFO ][o.e.p.PluginsService ] [ana-9ag] no plugins loaded
[2017-11-24T16:54:42,337][ERROR][o.e.b.Bootstrap ] Exception
java.lang.IllegalArgumentException: unknown setting [default.path.conf] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:293) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:256) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:135) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.node.Node.(Node.java:330) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.node.Node.(Node.java:245) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:212) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:322) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:130) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:121) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:69) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.0.0.jar:6.0.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.0.0.jar:6.0.0]

4

I get the same error for [default.path.data] and [default.path.logs] please check that any required plugins are installed, or check the breaking changes documentation for removed settings

But i do not have this configured in elasticsearch.yml anywhere...

5

Probably you didn’t update the startup scripts

6

Thanks for your reply. Can you please detail what needs to be changed in each file? Is manually hanging startup scripts always have to be done during upgrade procedure? I am following following documentation, point 4 for upgrading with RPM:
https://www.elastic.co/guide/en/elasticsearch/reference/current/rolling-upgrades.html

7

No I can’t. I’m afraid I don’t know about scripts and Linux packaging. May be @tlrx or @warkolm know?

8

I have changed the following ffile with modifications:
/usr/lib/systemd/system/elasticsearch.service

ExecStart=/usr/share/elasticsearch/bin/elasticsearch
-p ${PID_DIR}/elasticsearch.pid
--quiet
-Epath.logs=${LOG_DIR}
-Epath.data=${DATA_DIR} \

I also had to change the following in same file:

#Specifies the maximum number of processes
LimitNPROC=4096

Then I reloaded service and started Elasticsearch and all works well.
systemctl daemon-reload
systemctl start elasticsearch

Thanks for pointing me in right direction.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.