# java.lang.IllegalException: failed to obtain node locks

**URL:** <https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463>\
**Category:** Elasticsearch\
**Tags:** ilm-index-lifecycle-management\
**Created:** [September 6, 2019, 5:57pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463 "2019-09-06T17:57:56Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Matthew\_Graville](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_graville/32/53743_2.png) [@Matthew\_Graville](https://discuss.elastic.co/u/Matthew_Graville)\
**Post date:** [September 6, 2019, 5:57pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/1 "2019-09-06T17:57:56Z")

</div>

I am setting up ELK stack for the first time on 3 different servers respectively. Elasticsearch + Kibana are good (I think), and now onto logstash I am Parsing Logs following the guide:  
[https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html](https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html) @ the "Testing your pipeline" step. I realize I have no indicies or haven't done anything with the initial config for Elasticsearch:

`The date used in the index name is based on UTC, not the timezone where Logstash is running. If the query returns index_not_found_exception, make sure that logstash-$DATE reflects the actual name of the index. To see a list of available indexes, use this query: curl 'localhost:9200/_cat/indices?v'.`

So I go over the Elasticsearch server to create an Index for the first (also unsure on the PUT /indexname) API, so I figure I need to run elasticsearch from the bin before I use that command.  
`elasticsearch/bin$ ./elasticsearch`  
Error in [Main]:  
`java.lang.IllegalException: failed to obtain node locks, tried [/var/lib/elasticsearch] with lock id [0]; maybe these locations are note writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?`

Now based on the message, the /var location is writeable as I chmodded the errors earlier (I know I know bad practice, but dev). I don't know if I have started a node? Or how to go about increasing max nodes? Shutting down a running node?

Please keep in mind I am a n00b here, but have been making great progress and am just trying to follow the walkthrough. I am trying to add a index on elasticsearch for testing logstash.  
Many thanks to anyone who can help!

---

<div class="post-metadata">

**Author:** ![Doc\_Kaos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/doc_kaos/32/53671_2.png) [@Doc\_Kaos](https://discuss.elastic.co/u/Doc_Kaos)\
**Post date:** [September 6, 2019, 6:57pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/2 "2019-09-06T18:57:39Z")

</div>

By defaullt, ES will automatically create an index for you if it's not there. You can check /etc/elasticsearch/elasticsearch.yml for all your settings, specifically it can be set  
action.auto\_create\_index: logstash-\*  
Which will only allow indices with the name logstash-something to be auto-created

This is also where you would set node.max\_local\_storage\_nodes

To check if ES is already running just curl it:  
curl -XGET localhost:9200

---

<div class="post-metadata">

**Author:** ![Matthew\_Graville](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_graville/32/53743_2.png) [@Matthew\_Graville](https://discuss.elastic.co/u/Matthew_Graville)\
**Post date:** [September 6, 2019, 8:14pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/3 "2019-09-06T20:14:05Z")

</div>

I had to actually make it:  
`action.auto_create_index: ".security*,.monitoring*,..bunch more...,.ml*,logstash-*"`  
Still having some problems. Perhaps it needs to be .logstash-\* and not logstash-\*? Checking a few things.

Running curl `'<server>:9200/_cat/indices?v'` shows no indexes. Is there a way to physically add an index? i.e. how do I use that PUT command - like where - do I put "PUT"?

My thanks again for the help,

---

<div class="post-metadata">

**Author:** ![Matthew\_Graville](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_graville/32/53743_2.png) [@Matthew\_Graville](https://discuss.elastic.co/u/Matthew_Graville)\
**Post date:** [September 6, 2019, 8:17pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/4 "2019-09-06T20:17:26Z")

</div>

Actually the `'<server>:9200/_cat/indices?v'` works on the elasticsearch server, but not logstash server...I appear to no longer be able to just query 9200 either from logstash, ugh, so much troubleshooting lol!

---

<div class="post-metadata">

**Author:** ![Doc\_Kaos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/doc_kaos/32/53671_2.png) [@Doc\_Kaos](https://discuss.elastic.co/u/Doc_Kaos)\
**Post date:** [September 6, 2019, 8:24pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/5 "2019-09-06T20:24:38Z")

</div>

Unless your logstash server is running an instance of `elasticsearch` it won't work for localhost, and would need access to port 9200 on the elasticsearch server.  
If it works on the server running `elasticsearch` it could be a permissions/firewall/iptables problem

Yeah, common index's ES likes to create need to be whitelisted, I have `.security,.monitoring*,.watches,.triggered_watches,.watcher-history-*,.ml*, filebeat-*`

---

<div class="post-metadata">

**Author:** ![Matthew\_Graville](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_graville/32/53743_2.png) [@Matthew\_Graville](https://discuss.elastic.co/u/Matthew_Graville)\
**Post date:** [September 6, 2019, 8:32pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/6 "2019-09-06T20:32:58Z")

</div>

Thanks I got it (can see host:9200 from Logstash server). Still the auto index isn't working on logstash server putting in:  
`curl -XGET 'server:9200/logstash-$DATE/_search?pretty&q=response=200'`

Are you saying I also need to install elasticsearch on my logstash server or is it enough that it can just read/write the elasticsearch server?

---

<div class="post-metadata">

**Author:** ![Doc\_Kaos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/doc_kaos/32/53671_2.png) [@Doc\_Kaos](https://discuss.elastic.co/u/Doc_Kaos)\
**Post date:** [September 6, 2019, 8:40pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/7 "2019-09-06T20:40:59Z")

</div>

Just access to port 9200 should be enough.

Are you literally putting `$DATE` or replacing it with some date?

Try `curl -XGET server:9200/_cat/indices` to see if you have any index's yet.

---

<div class="post-metadata">

**Author:** ![Matthew\_Graville](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthew_graville/32/53743_2.png) [@Matthew\_Graville](https://discuss.elastic.co/u/Matthew_Graville)\
**Post date:** [September 6, 2019, 8:48pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/8 "2019-09-06T20:48:57Z")

</div>

> [@Doc\_Kaos](#):
>
> curl -XGET server:9200/\_cat/indices

THANK YOU SO MUCH. I got it, it turns out my it was logstash-$DATE-00001, so I recommend to everyone to see the indices first.

Thanks again Doc\_Kaos!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 4, 2019, 8:49pm UTC](https://discuss.elastic.co/t/java-lang-illegalexception-failed-to-obtain-node-locks/198463/9 "2019-10-04T20:49:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
