# Node.max\_local\_storage\_nodes was set to 3, now can't upgrade to 8.0 **URL:** \ **Category:** Elasticsearch\ **Created:** [April 18, 2022, 8:01pm UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642 "2022-04-18T20:01:35Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Justin\_Cross](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_cross/32/104500_2.png) [@Justin\_Cross](https://discuss.elastic.co/u/Justin_Cross)\ **Post date:** [April 18, 2022, 8:01pm UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/1 "2022-04-18T20:01:35Z")
Hi, So our node.max\_local\_storage\_nodes was set to 3 even though we only need 1 the whole time. Now we are trying to upgrade from 7.17.0 to 8 and it won't allow us even when we remove the node.max\_local\_storage\_nodes from Elasticsearch.yml. Is there a way to resolve this without losing the data? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 19, 2022, 7:13am UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/2 "2022-04-19T07:13:22Z")
> [@Justin\_Cross](#): > > Now we are trying to upgrade from 7.17.0 to 8 and it won't allow us even when we remove the node.max\_local\_storage\_nodes from Elasticsearch.yml "It won't allow us" is too vague to be able to give much help unfortunately. How is it not allowing you? Is it reporting an exception or other error? If so, would you share the exact message you're seeing? ---
**Author:** ![Justin\_Cross](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_cross/32/104500_2.png) [@Justin\_Cross](https://discuss.elastic.co/u/Justin_Cross)\ **Post date:** [April 19, 2022, 9:56am UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/3 "2022-04-19T09:56:53Z")
The error is java.lang.IllegalStateException: data path /usr/share/Elasticsearch/data/nodes cannot be upgraded automatically because it contains data from nodes with ordinals [0, 1], due to previous use of the now obsolete [node.max\_local\_storage\_nodes] setting. Please check the breaking changes docs for the current version of Elasticsearch to find an upgrade path node.max\_local\_storage\_nodes is no longer in Elasticsearch.yml ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 19, 2022, 10:41am UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/4 "2022-04-19T10:41:49Z")
Ah ok thanks that is clearer. It's complaining that `/usr/share/Elasticsearch/data/nodes/1` exists, which means that at some point in the past you did have multiple nodes running. Elasticsearch won't delete this directory since it can't tell that it is now obsolete. The safest fix would be to migrate all your nodes to a new data path (e.g. `/usr/share/Elasticsearch/data-2`) which will ensure that there's no data leftover from older nodes. ---
**Author:** ![Justin\_Cross](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_cross/32/104500_2.png) [@Justin\_Cross](https://discuss.elastic.co/u/Justin_Cross)\ **Post date:** [April 19, 2022, 5:39pm UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/5 "2022-04-19T17:39:00Z")
Removing the /nodes/1 directory allowed me to upgrade our test region but i didn't check if we had any data loss by removing that directory. I had saw the /0/ and /1/ both had the same indices folders. I don't know if this structure is caused by having replicas enabled or related to that node.max\_local\_storage\_nodes. We never ran multiple nodes on the same cluster so I was kind of thinking it is having these 2 folders due to replicas. Anyways when I do QA upgrade I will check if we lose any data when i remove the /nodes/1/ ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 19, 2022, 6:00pm UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/6 "2022-04-19T18:00:59Z")
The manual [strongly recommends not to modify things within the data path](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html#data-path). It might be fine, but it might also come back to haunt you later. I'd recommend migrating everything to a separate path instead. > [@Justin\_Cross](#): > > We never ran multiple nodes on the same cluster The only reason for Elasticsearch to create a `/nodes/1/` path is if there was already a node running against `/nodes/0/`. ---
**Author:** ![Justin\_Cross](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_cross/32/104500_2.png) [@Justin\_Cross](https://discuss.elastic.co/u/Justin_Cross)\ **Post date:** [April 19, 2022, 6:55pm UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/7 "2022-04-19T18:55:32Z")
Do you have a link for the proper way to migrate from one path to a new path? If I just change the path.data to a different path then it'll just be a new path with no data right. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 19, 2022, 7:20pm UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/8 "2022-04-19T19:20:19Z")
Yes the safe way to do this is to [add a new node](https://www.elastic.co/guide/en/elasticsearch/reference/current/add-elasticsearch-nodes.html) at the new data path, then [set a shard allocation filter](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-cluster.html#cluster-shard-allocation-filtering) to vacate the old node and wait for the shards to finish moving. Once that's done and everything is at `green` health you can safely remove the old data path. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 17, 2022, 7:21pm UTC](https://discuss.elastic.co/t/node-max-local-storage-nodes-was-set-to-3-now-cant-upgrade-to-8-0/302642/9 "2022-05-17T19:21:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.