Having issues installing ElasticSearch

Amazing · June 3, 2019, 9:37pm

I had 6x installed and something went wrong and it stopped working after it had been fine for months so I decided to go ahead and remove the 6x version and do a fresh install of 7.1.1. But so far I have not been able to get elasticsearch to stay running. When I run systemctl status elasticsearch I get the following output

Jun 03 17:32:33 server.servername.com elasticsearch[23772]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in ver...elease.
Jun 03 17:32:38 server.servername.com systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jun 03 17:32:38 server.servername.com systemd[1]: Unit elasticsearch.service entered failed state.
Jun 03 17:32:38 server.servername.com systemd[1]: elasticsearch.service failed.

I have not touched the configuration file yet so what am I doing wrong? Any help would be grateful!

warkolm · June 4, 2019, 12:34am

Please don't post pictures of text, they are difficult to read and some people may not be even able to see them.

Also please check your Elasticsearch logs.

Amazing · June 4, 2019, 1:23am

I see this in my logs

 [2019-06-03T17:32:38,372][ERROR][o.e.b.Bootstrap          ] [server.servername.com] Exception
java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or $
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:297) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.node.Node.<init>(Node.java:272) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) [elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-7.1.1.jar:7.1.1]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) [elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.1.1.jar:7.1.1]
Caused by: java.io.IOException: failed to obtain lock on /var/lib/elasticsearch/nodes/0
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:219) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-7.1.1.jar:7.1.1]
        ... 12 more
Caused by: java.nio.file.AccessDeniedException: /var/lib/elasticsearch/nodes/0/node.lock
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:182) ~[?:?]
        at java.nio.channels.FileChannel.open(FileChannel.java:292) ~[?:?]
        at java.nio.channels.FileChannel.open(FileChannel.java:345) ~[?:?]
        at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:125) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced761$
        at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 -$
        at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 -$
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:212) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-7.1.1.jar:7.1.1]
        ... 12 more
[2019-06-03T17:32:38,390][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [server.servername.com] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [$
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.1.1.jar:7.1.1]
        at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.1.1.jar:7.1.1]
Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not w$
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:297) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.node.Node.<init>(Node.java:272) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.1.jar:7.1.1]
        ... 6 more
Caused by: java.io.IOException: failed to obtain lock on /var/lib/elasticsearch/nodes/0
        at org.elasticsearch.env.NodeEnvironment$NodeLock.<init>(NodeEnvironment.java:219) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:267) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.node.Node.<init>(Node.java:272) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) ~[elasticsearch-7.1.1.jar:7.1.1]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.1.1.jar:7.1.1]
        ... 6 more

warkolm · June 4, 2019, 1:23am

Please format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you

Amazing · June 4, 2019, 1:32am

I edited it so it's easier to read.

warkolm · June 4, 2019, 1:37am

How did you install Elasticsearch? Is there a link to a guide/docs you used?

Amazing · June 4, 2019, 1:38am

https://www.elastic.co/guide/en/elasticsearch/reference/current/rpm.html
I installed using the repo

warkolm · June 4, 2019, 1:39am

What does ls -l /var/lib/elasticsearch/nodes/0 /var/lib/elasticsearch show?

Amazing · June 4, 2019, 1:41am

[root@private ~]#  ls -l /var/lib/elasticsearch/nodes/0 /var/lib/elasticsearch

/var/lib/elasticsearch:
total 4
drwxr-xr-x 3 987 985 4096 Oct 18 2018 nodes

/var/lib/elasticsearch/nodes/0:
total 8
drwxr-xr-x 7 987 985 4096 May 31 19:39 indices
-rw-r--r-- 1 987 985 0 Oct 18 2018 node.lock
drwxr-xr-x 2 987 985 4096 May 31 19:39 _state

warkolm · June 4, 2019, 1:43am

Ahh ok, so when you removed 6.X did you do a purge or just a remove? Cause it looks like there is still data there from the old install, and the user ID that it was using for the elasticsearch user is different to the one that 7.X is using.

If you don't want the data then you might want to rm the directory. If you do want it then do a chmod.

Amazing · June 4, 2019, 1:45am

When I removed it I used the yum remove elasticsearch command. If I did not do that correctly can you please tell me how to properly remove it?

warkolm · June 4, 2019, 1:51am

I think yum or rpm has an erase command? It's been a while since I used a RHEL based OS sorry, and my websearches aren't returning much useful.

Amazing · June 4, 2019, 1:52am

I just ran rm -rf /etc/elasticsearch after doing yum remove elasticsearch and now the directory and files are gone so I should be safe to try to reinstall now correct?

warkolm · June 4, 2019, 1:53am

If you've already removed the package, yep!

Amazing · June 4, 2019, 1:56am

That did not help and the same errors show.

java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch]] with lock id [0]; maybe these locations are not writable or $
    at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:297) ~[elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.node.Node.<init>(Node.java:272) ~[elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.node.Node.<init>(Node.java:252) ~[elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:211) ~[elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:325) [elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-7.1.1.jar:7.1.1]
    at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.1.1.jar:7.1.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) [elasticsearch-7.1.1.jar:7.1.1]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-7.1.1.jar:7.1.1]

I have even removed the old java installs that I had before cause I had read somewhere that java is now bundled with ES. Also I tried to remove ES again and ran the same command to delete the folder but then I thought to run ls -l /var/lib/elasticsearch/nodes/0 /var/lib/elasticsearch
and got this again

`[root@private ~]# rm -rf /etc/elasticsearch
  [root@private ~]# ls -l /var/lib/elasticsearch/nodes/0 /var/lib/elasticsearch
  /var/lib/elasticsearch:
  total 4
  drwxr-xr-x 3 987 985 4096 Oct 18  2018 nodes

 /var/lib/elasticsearch/nodes/0:
 total 8
 drwxr-xr-x 7 987 985 4096 May 31 19:39 indices
-rw-r--r-- 1 987 985    0 Oct 18  2018 node.lock
 drwxr-xr-x 2 987 985 4096 May 31 19:39 _state`

Amazing · June 4, 2019, 2:59am

I figured it out ! Still was left over files even after doing that so I ran

rm -rf  /var/lib/elasticsearch/nodes
rm -rf  /var/lib/elasticsearch/

And Finally ran

ls -l /var/lib/elasticsearch/nodes/0 /var/lib/elasticsearch

And finally got

ls: cannot access /var/lib/elasticsearch/nodes/0: No such file or directory

So then I reinstalled 7.1.1 and it now stays running. I have one more question though what else do I need to install for this to work the best and to be secure? I am using this for a Xenforo forum and I wanted to ensure all is safe and secure. Also thanks so much for your help.

warkolm · June 4, 2019, 3:01am

I don't know Xenforo (and would highly recommend using Discourse - which is what this platform is), but if you are using 7.1 then make sure you are using the default distribution and setup TLS and security

Amazing · June 4, 2019, 3:06am

Do you have any links for how to setup what you had suggested ? I want to ensure I am doing everything right. Also do I need to add to the config file anything to enable x-pack?

nuwan · June 4, 2019, 5:16am

nuwan · June 4, 2019, 5:16am

or http://www.icode9.com/content-3-129394.html

Topic		Replies	Views
ES doesn't start Elasticsearch	4	4910	August 9, 2018
Elasticsearch - error while trying to start service on OpenShift Elasticsearch	8	794	January 6, 2021
Not able to run elasticsearch Elasticsearch	2	558	January 1, 2020
Elasticsearch installation Elasticsearch	46	3787	November 5, 2021
Troubles with docker compose + image elasticsearch:6.8.0 Elasticsearch docker	1	732	March 27, 2020

Having issues installing ElasticSearch

Related topics