# Node with zero shards fails to start with error "Node is started with node.data=false, but has shard data"

**URL:** <https://discuss.elastic.co/t/node-with-zero-shards-fails-to-start-with-error-node-is-started-with-node-data-false-but-has-shard-data/205450>\
**Category:** Elasticsearch\
**Created:** [October 28, 2019, 11:37am UTC](https://discuss.elastic.co/t/node-with-zero-shards-fails-to-start-with-error-node-is-started-with-node-data-false-but-has-shard-data/205450 "2019-10-28T11:37:08Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![tomhe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomhe/32/120065_2.png) [@tomhe](https://discuss.elastic.co/u/tomhe)\
**Post date:** [October 28, 2019, 11:37am UTC](https://discuss.elastic.co/t/node-with-zero-shards-fails-to-start-with-error-node-is-started-with-node-data-false-but-has-shard-data/205450/1 "2019-10-28T11:37:08Z")

</div>

I'm trying to remove the data role from one node by first excluding the node from shard allocation with:

```
PUT _cluster/settings
{
  "transient" : {
    "cluster.routing.allocation.exclude._name" : "node-001"
  }
}

```

When the node showed 0 shards I changed "node.data" to false and restarted it, but the node failed to start with this error:

```
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Node is started with node.data=false, but has shard data: [/usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/4, /usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/7, /usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/2, /usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/9, /usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/3, /usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/1, /usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/8, /usr/share/elasticsearch/data/nodes/0/indices/KdyBcBhwQhmrcvW9mPy5eQ/6, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/4, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/2, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/0, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/9, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/3, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/1, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/5, /usr/share/elasticsearch/data/nodes/0/indices/YJXzvwdRR1qU6I23pr0SXQ/6, /usr/share/elasticsearch/data/nodes/0/indices/bagM1fomTDm_dwV3mymIpg/4, /usr/share/elasticsearch/data/nodes/0/indices/bagM1fomTDm_dwV3mymIpg/2, /usr/share/elasticsearch/data/nodes/0/indices/bagM1fomTDm_dwV3mymIpg/0, /usr/share/elasticsearch/data/nodes/0/indices/bagM1fomTDm_dwV3mymIpg/3]. Use 'elasticsearch-node repurpose' tool to clean up

```

There are no shards allocated to his node, so why am I getting this error?

---

<div class="post-metadata">

**Author:** ![tomhe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomhe/32/120065_2.png) [@tomhe](https://discuss.elastic.co/u/tomhe)\
**Post date:** [October 28, 2019, 12:21pm UTC](https://discuss.elastic.co/t/node-with-zero-shards-fails-to-start-with-error-node-is-started-with-node-data-false-but-has-shard-data/205450/2 "2019-10-28T12:21:31Z")

</div>

I found the cause: There were shards from three closed indices on that node.

---

<div class="post-metadata">

**Author:** ![tomhe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tomhe/32/120065_2.png) [@tomhe](https://discuss.elastic.co/u/tomhe)\
**Post date:** [October 28, 2019, 2:07pm UTC](https://discuss.elastic.co/t/node-with-zero-shards-fails-to-start-with-error-node-is-started-with-node-data-false-but-has-shard-data/205450/3 "2019-10-28T14:07:00Z")

</div>

Ran into the same situation again on the next node that I no longer wanted to use as a data node. The node has zero shards allocated.

```
sh-4.2# bin/elasticsearch-node repurpose -v
------------------------------------------------------------------------

    WARNING: Elasticsearch MUST be stopped before running this tool.

Obtaining lock for node
Collecting shard data paths
Paths to clean up:
  /usr/share/elasticsearch/data/nodes/0/indices/myrWs3pnSmuZwTplN3XKdw/3
Indices affected:
  important_index_in_use
Found 1 shards in 1 indices to clean up
Node is being re-purposed as master and no-data. Clean-up of shard data will be performed.
Do you want to proceed?
Confirm [y/N]

```

(Index is not named "important\_index\_in\_use", but I changed it here in the post.)

This index is in use. Why am I seeing shard data here even though the node has zero allocated shards?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [October 28, 2019, 2:35pm UTC](https://discuss.elastic.co/t/node-with-zero-shards-fails-to-start-with-error-node-is-started-with-node-data-false-but-has-shard-data/205450/4 "2019-10-28T14:35:10Z")

</div>

Elasticsearch cleans up unused shard data from a data node when the shard is fully allocated (i.e. `green` health) none of its shards are relocating. If that wasn't the case when this node was shut down then the data might not have been cleaned up. If this index reports `green` health then you can safely remove the shard data on this node.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 25, 2019, 2:35pm UTC](https://discuss.elastic.co/t/node-with-zero-shards-fails-to-start-with-error-node-is-started-with-node-data-false-but-has-shard-data/205450/5 "2019-11-25T14:35:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
