Unable to Start ElasticSearch 5.6.16

#1

I just downgraded from v6.5.4 to v5.6.16 to meet the requirements of a web app I am installing.

When I start the service (service elasticsearch start) it says OK, but when I check the status (service elasticsearch start), I get the following:

[root@ctl elasticsearch]# service elasticsearch status
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2019-07-29 18:02:53 CDT; 11min ago
Docs: http://www.elastic.co
Process: 19878 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p {PID_DIR}/elasticsearch.pid --quiet -Edefault.path.logs={LOG_DIR} -Edefault.path.data={DATA_DIR} -Edefault.path.conf={CONF_DIR} (code=exited, status=1/FAILURE)
Process: 19876 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
Main PID: 19878 (code=exited, status=1/FAILURE)

Jul 29 18:02:53 ctl elasticsearch[19878]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123)
Jul 29 18:02:53 ctl elasticsearch[19878]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70)
Jul 29 18:02:53 ctl elasticsearch[19878]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134)
Jul 29 18:02:53 ctl elasticsearch[19878]: at org.elasticsearch.cli.Command.main(Command.java:90)
Jul 29 18:02:53 ctl elasticsearch[19878]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91)
Jul 29 18:02:53 ctl elasticsearch[19878]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84)
Jul 29 18:02:53 ctl elasticsearch[19878]: Refer to the log for complete error details.
Jul 29 18:02:53 ctl systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jul 29 18:02:53 ctl systemd[1]: Unit elasticsearch.service entered failed state.
Jul 29 18:02:53 ctl systemd[1]: elasticsearch.service failed

#2

Hey John,

the fourth last logging line states that you should refer to the log for complete error details. Have you checked the elasticsearch log in /var/log/elasticsearch? Is there any additional info available that might help to debug this issue further?

--Alex

#3

Nothing is entered in the log file.

#4

ok, then it is likely that using journalctl you can get far more information than just the few lines above... can you try using it? You can also run journalctl -f (IIRC) and then try to start Elasticsearch.

#5

@spinscale

I was finally able to get this information from the logs

Aug 05 15:52:13 ctl elasticsearch[22288]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: org.apache.lucene.index.IndexFormatTooNewException: Format version is not supported (resource BufferedChecksumIndexInput(SimpleFSIndexInput(path="/etc/elasticsearch/elasticsearch.keystore"))): 3 (needs to be between 1 and 2)

I deleted and recreated the keystore file and now receive the following from
service elasticsearch status

● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2019-08-05 16:24:26 CDT; 102ms ago
Docs: http://www.elastic.co
Process: 23585 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p {PID_DIR}/elasticsearch.pid --quiet -Edefault.path.logs={LOG_DIR} -Edefault.path.data={DATA_DIR} -Edefault.path.conf={CONF_DIR} (code=exited, status=1/FAILURE)
Process: 23583 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
Main PID: 23585 (code=exited, status=1/FAILURE)

Aug 05 16:24:21 ctl systemd[1]: Starting Elasticsearch...
Aug 05 16:24:21 ctl systemd[1]: Started Elasticsearch.
Aug 05 16:24:26 ctl systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Aug 05 16:24:26 ctl systemd[1]: Unit elasticsearch.service entered failed state.
Aug 05 16:24:26 ctl systemd[1]: elasticsearch.service failed.

There is nothing in the elasticsearch.log file nor anything pointing to the error using journalctl