# Unable to start non-data nodes Elasticsearch after upgrading from 6.42 to 6.6.1 **URL:** \ **Category:** Elasticsearch\ **Created:** [February 21, 2019, 9:32am UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356 "2019-02-21T09:32:26Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Hari\_Prasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_prasad/32/56784_2.png) [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Post date:** [February 21, 2019, 9:32am UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/1 "2019-02-21T09:32:27Z")
I upgraded my cluster based on the instructions from the [link](https://www.elastic.co/guide/en/elasticsearch/reference/current/rolling-upgrades.html) After upgrading none of the non-data nodes started. All of them had the below error [2019-02-20T22:57:35,922][WARN][o.e.b.ElasticsearchUncaughtExceptionHandler] [node1] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to obtain node locks, tried [[/etc/elasticsearch/data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max\_local\_storage\_nodes] (was [1])? I even tried setting **node.max\_local\_storage\_nodes** the value to 5 and started and still the error persisted. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 21, 2019, 9:44am UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/2 "2019-02-21T09:44:26Z")
How many nodes are running locally? ---
**Author:** ![Hari\_Prasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_prasad/32/56784_2.png) [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Post date:** [February 21, 2019, 9:55am UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/3 "2019-02-21T09:55:08Z")
only 1 ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 21, 2019, 10:06am UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/4 "2019-02-21T10:06:08Z")
Then what is the problem if the node is already running? Why do you want to start another one? ---
**Author:** ![Hari\_Prasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_prasad/32/56784_2.png) [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Post date:** [February 21, 2019, 11:54am UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/5 "2019-02-21T11:54:12Z")
I had installed the Elasticsearch via yum install so i have only one Elasticsearch running. This error is appearing when I do a **service elasticsearch restart** ---
**Author:** ![Hari\_Prasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_prasad/32/56784_2.png) [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Post date:** [February 21, 2019, 12:01pm UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/6 "2019-02-21T12:01:13Z")
Pasting the full stack trace ``` [2019-02-21T03:58:35,885][WARN][o.e.b.ElasticsearchUncaughtExceptionHandler] [node1] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.6.1.jar:6.6.1] at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.6.1.jar:6.6.1] Caused by: java.lang.IllegalStateException: Failed to create node environment at org.elasticsearch.node.Node.(Node.java:298) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.node.Node.(Node.java:265) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.1.jar:6.6.1] ... 6 more Caused by: java.nio.file.AccessDeniedException: /etc/elasticsearch/data/nodes/0 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?] at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:?] at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_181] at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_181] at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_181] at org.elasticsearch.env.NodeEnvironment.lambda$new$0(NodeEnvironment.java:270) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.env.NodeEnvironment$NodeLock.(NodeEnvironment.java:203) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.env.NodeEnvironment.(NodeEnvironment.java:267) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.node.Node.(Node.java:295) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.node.Node.(Node.java:265) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.6.1.jar:6.6.1] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.6.1.jar:6.6.1] ... 6 more ^C [node1:/home/users/usr1]# ps aux |grep java root 31871 0.0 0.0 112676 728 pts/0 R+ 03:58 0:00 grep --color=auto java [node1:/home/users/usr1]# ``` ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 21, 2019, 12:09pm UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/7 "2019-02-21T12:09:07Z")
The error here does not seem to be the same as you mentioned initially: ```auto Caused by: java.nio.file.AccessDeniedException: /etc/elasticsearch/data/nodes/0 ``` Most likely a permission issue? ---
**Author:** ![Hari\_Prasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_prasad/32/56784_2.png) [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Post date:** [February 21, 2019, 12:40pm UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/8 "2019-02-21T12:40:25Z")
Thank you @dadoonet The permission was > drwxr-sr-x 2 root elasticsearch 6 Feb 21 01:08 nodes updating to **elasticsearch:elasticsearch** started the node. **But why did change happen during upgrade and that too only in non-data nodes. Is there any special thing that I need to take care while upgrading.** ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [February 21, 2019, 1:06pm UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/9 "2019-02-21T13:06:40Z")
I don't know. May be you ran something as root instead of elasticsearch at some point? ---
**Author:** ![Hari\_Prasad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hari_prasad/32/56784_2.png) [@Hari\_Prasad](https://discuss.elastic.co/u/Hari_Prasad)\ **Post date:** [February 25, 2019, 1:06pm UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/10 "2019-02-25T13:06:10Z")
@dadoonet but the steps to upgrade was run via script it was same on all the nodes including the data nodes. The data nodes are fine post upgrade. I am just trying to see that if I need to make any specific preparation for my prod setup. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 25, 2019, 1:19pm UTC](https://discuss.elastic.co/t/unable-to-start-non-data-nodes-elasticsearch-after-upgrading-from-6-42-to-6-6-1/169356/11 "2019-03-25T13:19:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.