# Elasticsearch 6.0: failed to obtain node locks

**URL:** <https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994>\
**Category:** Elasticsearch\
**Created:** [December 11, 2017, 3:38am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994 "2017-12-11T03:38:28Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![lanlin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lanlin/32/25269_2.png) [@lanlin](https://discuss.elastic.co/u/lanlin)\
**Post date:** [December 11, 2017, 3:38am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/1 "2017-12-11T03:38:28Z")

</div>

failed to start multi instance on one server after upgrade from 5.x to 6.0.

system: centos 7.4  
elasticsearch: 6.0.1

i use the systemd to start elasticsearch, one master node and two data node. it works fine under 5.x version, but failed in 6.0.

Here's the logs

```
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: [2017-12-11T10:59:05,646][INFO][o.e.n.Node] [] initializing ...
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: [2017-12-11T10:59:05,687][WARN][o.e.b.ElasticsearchUncaughtExceptionHandler] [] uncaught exception in thread [main]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/ela
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:134) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:121) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:69) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: Caused by: java.lang.IllegalStateException: failed to obtain node locks, tried [[/var/lib/elasticsearch/elasticsearch]] with loc
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:244) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.node.Node.<init>(Node.java:263) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.node.Node.<init>(Node.java:245) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:322) ~[elasticsearch-6.0.1.jar:6.0.1]
Dec 11 10:59:05 caf3c89c5314 elasticsearch[1088]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:130) ~[elasticsearch-6.0.1.jar:6.0.1]

```

it seems that the "path.data" config not working. All three noses still using /var/lib/elasticsearch/elasticsearch as the data path.

how to resolve this?

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [December 15, 2017, 1:13pm UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/2 "2017-12-15T13:13:36Z")

</div>

Would you please share the contents of:

- `/etc/elasticsearch/elasticsearch.yml`
- `/etc/default/elasticsearch`
- the initial log line that shows `JVM arguments`

---

<div class="post-metadata">

**Author:** ![lanlin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lanlin/32/25269_2.png) [@lanlin](https://discuss.elastic.co/u/lanlin)\
**Post date:** [December 19, 2017, 9:32am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/3 "2017-12-19T09:32:01Z")

</div>

Sorry, late for my reply. I'v aready sovled this problem.

After install elasticsearch with RPM in centos 7, the default elasticsearch.service can only start one instance. So i create another service files like below to start multiple instances.

All services has it's own yml, jvm.options, log4j2.properties and instance.service file.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/6/262210238c6cd32a27ca03cc205d70ba4a808f93.png)

As you see, stam0 is one of the service files.

After i open the "elasticsearch-env" bin file, i found the reason.

path "/usr/share/elasticsearch/bin/elasticsearch-env"

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/d/7d666128a647c98f012d0586fec2c57e7c410f37.png)

No matter how you setting your configs, the "elasticsearch-env" aways load the default  
config from "/etc/sysconfig/elasticsearch", your custom configs been override.

Finally, i remove /etc/sysconfig/elasticsearch and all my instances started.

Thanks anyway.

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [December 19, 2017, 1:08pm UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/4 "2017-12-19T13:08:13Z")

</div>

You should not edit the scripts. Instead, only put environment variables in /etc/sysconfig/elasticsearch that can be shared across instances. For environment variable overrides otherwise, use instance-specific unit overrides files.

---

<div class="post-metadata">

**Author:** ![lanlin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lanlin/32/25269_2.png) [@lanlin](https://discuss.elastic.co/u/lanlin)\
**Post date:** [December 20, 2017, 2:29am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/5 "2017-12-20T02:29:28Z")

</div>

Well, you may misunderstand me. I didn't edit the scripts, i just run a simple remove command

```
sudo rm -rf /etc/sysconfig/elasticsearch

```

After that the "elasticsearch-env" can't find the default configs, so my own configs not been override by the default configs.

If i don't do that, only one instance can be started, not all.

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [December 20, 2017, 2:49am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/6 "2017-12-20T02:49:57Z")

</div>

I understood but I misspoke. I meant you should not remove the `/etc/sysconfig/elasticsearch` script (note: it's a script for sourcing environment variables in the `elasticsearch-env` script), in the future the check for existence will be removed and your process will fail to start without it. Instead, leave it present with shared environment variables (e.g., remove `ES_PATH_CONF`).

---

<div class="post-metadata">

**Author:** ![lanlin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lanlin/32/25269_2.png) [@lanlin](https://discuss.elastic.co/u/lanlin)\
**Post date:** [December 20, 2017, 4:09am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/7 "2017-12-20T04:09:11Z")

</div>

Ok, thanks. Next time i will try the way that you suggested.😀

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [December 20, 2017, 10:14am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/8 "2017-12-20T10:14:56Z")

</div>

You're welcome, and sorry for the misspeak, I am sure that was confusing.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 17, 2018, 10:15am UTC](https://discuss.elastic.co/t/elasticsearch-6-0-failed-to-obtain-node-locks/110994/9 "2018-01-17T10:15:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
