# Shard failure during kibana visualisation and 'Discover'

**URL:** <https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806>\
**Category:** Elasticsearch\
**Created:** [May 20, 2019, 12:11pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806 "2019-05-20T12:11:40Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![elk11](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@elk11](https://discuss.elastic.co/u/elk11)\
**Post date:** [May 20, 2019, 12:11pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/1 "2019-05-20T12:11:40Z")

</div>

Hi,

I'm upgrading elasticsearch cluster from 5.x to 7.0. I have setup a 7.0.0 cluster and then reindexed data from 5.x cluster. And then exported all the saved objects from old cluster to new one.

post this, I'm getting error of 'x of y shards failed' when trying to view some visualisation or search on some indexes.

There's nothing about this failure on elasticsearch logs. So how should i fix this error?

Thanks for the help!

---

<div class="post-metadata">

**Author:** ![elk11](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@elk11](https://discuss.elastic.co/u/elk11)\
**Post date:** [May 21, 2019, 6:39pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/2 "2019-05-21T18:39:11Z")

</div>

Can somebody help me with this? There's nothing in elasticsearch logs. And all shards are 'green' clsuter status is green. I don't know how to proceed with this.

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [May 21, 2019, 8:47pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/3 "2019-05-21T20:47:02Z")

</div>

> [@elk11](#):
>
> I have setup a 7.0.0 cluster and then reindexed data from 5.x cluster.

Ordinarily our upgrade procedures are designed to take you from the last minor release of a major to the next major e.g 6.7 to 7. I suspect this is an issue with the format of the saved 5.x kibana visualisations and the format required by 7. Probably best to ask in the Kibana forum

---

<div class="post-metadata">

**Author:** ![elk11](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@elk11](https://discuss.elastic.co/u/elk11)\
**Post date:** [May 22, 2019, 11:10am UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/4 "2019-05-22T11:10:22Z")

</div>

Thanks for the reply @Mark_Harwood

Yes I'll ask it in Kibana forum too. But this problem is happening on 'Discover' tab too. That's what led me think that it might be an elasticsearch issue too.

What do you think about shard failure problem showing up on 'Discover' tab?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [May 22, 2019, 11:49am UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/5 "2019-05-22T11:49:53Z")

</div>

Is it possible you have mixed es versions in your cluster?  
Hard to say what is going on without seeing details of the failures I’m afraid

---

<div class="post-metadata">

**Author:** ![elk11](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@elk11](https://discuss.elastic.co/u/elk11)\
**Post date:** [May 22, 2019, 12:09pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/6 "2019-05-22T12:09:36Z")

</div>

No, All nodes are 7.0.0. And cluster status is green.

```
{
  "cluster_name" : "elasticsearch_cluster",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 5,
  "number_of_data_nodes" : 5,
  "active_primary_shards" : 643,
  "active_shards" : 1286,
  "relocating_shards" : 2,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue" : "0s",
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent" : "100.0%",
  "active_shards_percent_as_number" : 100.0
}

```

And there's no clue about the error in neither kibana or elasticsearch logs.

---

<div class="post-metadata">

**Author:** ![elk11](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@elk11](https://discuss.elastic.co/u/elk11)\
**Post date:** [May 22, 2019, 12:19pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/7 "2019-05-22T12:19:49Z")

</div>

I'm running both Elasticsearch and kibana in docker containers. So i'm checking docker logs of those containers. Is there anywhere else I can get the info about these failures?

---

<div class="post-metadata">

**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\
**Post date:** [May 22, 2019, 12:33pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/8 "2019-05-22T12:33:17Z")

</div>

It’s possible to change logging levels to see if that helps but I think my next step would be to simplify to the smallest reproducible example eg first try take Kibana out of the equation and run the query it runs via curl. Then reduce the number of indices you’re searching and then number of query clauses etc

---

<div class="post-metadata">

**Author:** ![elk11](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@elk11](https://discuss.elastic.co/u/elk11)\
**Post date:** [May 22, 2019, 12:37pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/9 "2019-05-22T12:37:32Z")

</div>

Thanks Mark, I'll try that.

---

<div class="post-metadata">

**Author:** ![hogbinj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hogbinj/32/14744_2.png) [@hogbinj](https://discuss.elastic.co/u/hogbinj)\
**Post date:** [May 24, 2019, 5:38pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/10 "2019-05-24T17:38:56Z")

</div>

I had it where I was using a wild card in the Kibana index pattern to pick up multiple indicies

e.g

index -\> alias  
data\_type1\_001 -\> MYCO\_DATA\_type1  
data\_type1\_002 -\> MYCO\_DATA\_type1  
data\_type2\_001 -\> MYCO\_DATA\_type2

For MYCO\_Data\_type1 the field definitions in the data\_type1\_001 and data\_type1\_002 indices need to be the same

For MYCO\_Data\* the field definitions in the the data\_type1\_001 and data\_type1\_002 AND data\_type2\_001 indices need to be the same

Make sure you haven't pre defined one field as a keyword and then in the other index where you haven't ES has done the text with sub key keyword

Mapping from template:

```auto
{
  "data_type1_001" : {
    "aliases" : {
      "MYCO_DATA_type1" : { }
    },
    "mappings" : {
      "properties" : {
        "meta_data_object" : {
          "properties" : {
            "meta_data_object_field_1" : {
              "type" : "keyword"
            },
[...]

```

Mapping that happens when ES "guesses"

```auto
{
  "data_type1_002" : {
    "aliases" : {
      "MYCO_DATA_type1" : { }
    },
    "mappings" : {
      "properties" : {
        "meta_data_object" : {
          "properties" : {
            "meta_data_object_field_1" : {
              "type" : "text",
              "fields" : {
                "keyword" : {
                  "type" : "keyword",
                  "ignore_above" : 256
                }
              }
            },
[...]
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 21, 2019, 5:38pm UTC](https://discuss.elastic.co/t/shard-failure-during-kibana-visualisation-and-discover/181806/11 "2019-06-21T17:38:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
