Elasticsearch failed to start yesterday

Elastic Stack Elasticsearch
21

Yes I will, thanks for your help :slight_smile:

22

Linux Version: Ubuntu 16.04
Elasticsearch Version: 1.7.3

I have recently added some new nodes to my existing cluster, and randomly one server would not start elasticsearch on restart. When manually trying to start it gives the following error:

>sudo systemctl status elasticsearch


* elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2017-06-16 12:32:28 UTC; 3s ago
     Docs: http://www.elastic.co
  Process: 1437 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.pidfile=$PID_DIR/elasticsearch.pid -Des.default.path.home=$ES_HOME -Des.default.p
 Main PID: 1437 (code=exited, status=3)

Jun 16 12:32:28 els02.jobsoid.net elasticsearch[1437]: java.io.FileNotFoundException: /var/run/elasticsearch/elasticsearch.pid (No such file or directory)
Jun 16 12:32:28 els02.jobsoid.net elasticsearch[1437]:         at java.io.FileOutputStream.open0(Native Method)
Jun 16 12:32:28 els02.jobsoid.net elasticsearch[1437]:         at java.io.FileOutputStream.open(FileOutputStream.java:270)
Jun 16 12:32:28 els02.jobsoid.net elasticsearch[1437]:         at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
Jun 16 12:32:28 els02.jobsoid.net elasticsearch[1437]:         at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
Jun 16 12:32:28 els02.jobsoid.net elasticsearch[1437]:         at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:194)
Jun 16 12:32:28 els02.jobsoid.net elasticsearch[1437]:         at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
Jun 16 12:32:28 els02.jobsoid.net systemd[1]: elasticsearch.service: Main process exited, code=exited, status=3/NOTIMPLEMENTED
Jun 16 12:32:28 els02.jobsoid.net systemd[1]: elasticsearch.service: Unit entered failed state.
Jun 16 12:32:28 els02.jobsoid.net systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

If I create a folder /var/run/elasticsearch and change ownership to elasticsearch:elasticsearch then elasticsearch starts. On restart again the same issue.

I tried changing the PID_DIR folder to /var/run in the service file at /usr/lib/systemd/system/elasticsearch.service
with that it gives an error: java.io.FileNotFoundException: /var/run/elasticsearch.pid (Permission denied) instead of "no such file or directory"

One other thing i noticed, when I manually create the dir in /var/run and force elasticsearch to start, it seems to work fine but the GET /_nodes does not return any stats of the OS or Filesystem.

Makes me think its some permission issue?

23

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.