# Unassigned shards on ES 2.4.4 **URL:** \ **Category:** Elasticsearch\ **Created:** [July 27, 2018, 11:21am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927 "2018-07-27T11:21:37Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Jordi\_Cabre](https://avatars.discourse-cdn.com/v4/letter/j/47e85d/32.png) [@Jordi\_Cabre](https://discuss.elastic.co/u/Jordi_Cabre)\ **Post date:** [July 27, 2018, 11:21am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/1 "2018-07-27T11:21:37Z")
I'm using an ES 2.4.4. Currently, I'm getting this content response at `_cluster/health?level=nodes` ``` { "cluster_name":"logging-es", "status":"red", "timed_out":false, "number_of_nodes":1, "number_of_data_nodes":1, "active_primary_shards":62, "active_shards":62, "relocating_shards":0, "initializing_shards":0, "unassigned_shards":2, "delayed_unassigned_shards":0, "number_of_pending_tasks":0, "number_of_in_flight_fetch":0, "task_max_waiting_in_queue_millis":0, "active_shards_percent_as_number":96.875 } ``` As you can see, cluster has two shards unassigned: `"unassigned_shards":2`. Related indices are: ``` $ _cat/indices -s | grep red red open project.istio-project-two.242ef609-8f... 1 0 red open project.3scale-amp-2.08baf3f... 1 0 ``` These are the unassigned shards: ``` $ _cat/shards | grep UNASSIGNED project.istio-project-two.242ef609-8f... 0 p UNASSIGNED project.3scale-amp-2.08baf3f5-c3... 0 p UNASSIGNED ``` I don't know which is the reason ES is running in this issue. Every day I delete these ones and then are created again. However, every day morning I need to apply this workaround I don't feel confortable. The number of nodes: ``` $ _cat/nodes 10.128.2.21 10.128.2.21 67 99 0.41 d * logging-es-data-master-yl7ddqw7 ``` Index settings: ``` $GET /project.istio-project-two.242ef609-8f/_settings?pretty { "project.istio-project-two.242ef609-8f..." : { "settings" : { "index" : { "creation_date" : "1532649602432", "refresh_interval" : "5s", "number_of_shards" : "1", "number_of_replicas" : "0", "uuid" : "XY-tFCkhQVOLlXmndoKpSQ", "version" : { "created" : "2040499" } } } } } ``` Any ideas? ---
**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\ **Post date:** [July 27, 2018, 11:29am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/2 "2018-07-27T11:29:10Z")
This could be caused by dangling indices. Check your logs on the nodes to see if there are any warnings about dangling indices. ---
**Author:** ![Jordi\_Cabre](https://avatars.discourse-cdn.com/v4/letter/j/47e85d/32.png) [@Jordi\_Cabre](https://discuss.elastic.co/u/Jordi_Cabre)\ **Post date:** [July 27, 2018, 11:38am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/3 "2018-07-27T11:38:15Z")
> [@ywelsch](#): > > dangling indices I've took a look but I¡ve not been able to found any related issue about "dangling" ---
**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\ **Post date:** [July 27, 2018, 5:58pm UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/4 "2018-07-27T17:58:30Z")
[Can you provide the routing table](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-state.html) for that particular index with the unassigned shards? `GET /_cluster/state/routing_table/your_index_name_that_has_unassigned_shards` Note that in ES 5.x and above we have the allocation explain API that provides a lot of details on why a shard is unassigned. So maybe that's also a good motivation to upgrade your cluster? 🙂 ---
**Author:** ![Jordi\_Cabre](https://avatars.discourse-cdn.com/v4/letter/j/47e85d/32.png) [@Jordi\_Cabre](https://discuss.elastic.co/u/Jordi_Cabre)\ **Post date:** [July 30, 2018, 7:36am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/5 "2018-07-30T07:36:13Z")
Here it goes. It seems to provide more related information: ``` $ curl --key /etc/elasticsearch/secret/admin-key --cert /etc/elasticsearch/secret/admin-cert --caouting_table/project.istio-project.0427311d-8f11-11e8-9139-0050569fe304.2018.07.30?pretty -s { "cluster_name" : "logging-es", "routing_table" : { "indices" : { "project.istio-project.0427311d-8f11-11e8-9139-0050569fe304.2018.07.30" : { "shards" : { "0" : [ { "state" : "UNASSIGNED", "primary" : true, "node" : null, "relocating_node" : null, "shard" : 0, "index" : "project.istio-project.0427311d-8f11-11e8-9139-0050569fe304.2018.07.30", "version" : 3, "unassigned_info" : { "reason" : "ALLOCATION_FAILED", "at" : "2018-07-30T02:02:29.633Z", "details" : "engine failure, reason [refresh failed], failure CorruptIndexException[codec footer mismatch (file truncated?): actual footer=135071757 vs expected footer=-1071082520 (resource=NIOFSIndexInput(path=\"/elasticsearch/persistent/logging-es/data/logging-es/nodes/0/indices/project.istio-project.0427311d-8f11-11e8-9139-0050569fe304.2018.07.30/0/index/_12v.cfs\") [slice=_12v_Lucene50_0.pos])]" } } ] } } } } } ``` ---
**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\ **Post date:** [July 30, 2018, 7:53am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/6 "2018-07-30T07:53:39Z")
this looks like disk corruption. If you have backups, now's a good time to restore them. ---
**Author:** ![Jordi\_Cabre](https://avatars.discourse-cdn.com/v4/letter/j/47e85d/32.png) [@Jordi\_Cabre](https://discuss.elastic.co/u/Jordi_Cabre)\ **Post date:** [July 30, 2018, 8:13am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/7 "2018-07-30T08:13:37Z")
Could you suggest me any way to dig into this issue? I think it's an issue related with glusterFS. We are mounting ES data persistence using a GlusterFS volume. I would like to dig a bit more into this issue. Could you suggest me any strategy no solve that? Any ideas will be welcome. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [July 30, 2018, 8:15am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/8 "2018-07-30T08:15:50Z")
I would recommend avoiding GlusterFS as it is not supported and there are known issues. See [this thread](https://discuss.elastic.co/t/underlying-file-changed-by-an-external-force/137555/4) for further details. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 27, 2018, 8:15am UTC](https://discuss.elastic.co/t/unassigned-shards-on-es-2-4-4/141927/9 "2018-08-27T08:15:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.