# Shard size is different between primary and replica

**URL:** <https://discuss.elastic.co/t/shard-size-is-different-between-primary-and-replica/150079>\
**Category:** Elasticsearch\
**Created:** [September 26, 2018, 7:58pm UTC](https://discuss.elastic.co/t/shard-size-is-different-between-primary-and-replica/150079 "2018-09-26T19:58:28Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dwilches](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dwilches/32/48586_2.png) [@dwilches](https://discuss.elastic.co/u/dwilches)\
**Post date:** [September 26, 2018, 7:58pm UTC](https://discuss.elastic.co/t/shard-size-is-different-between-primary-and-replica/150079/1 "2018-09-26T19:58:28Z")

</div>

We have an index that is divided in 10 primary shards (with 1 replica per shard, so we have 20 in total).

I expected the shards to have roughly the same size, so I plotted a histogram with them:

![image](https://us1.discourse-cdn.com/elastic/original/3X/4/4/4439d23ac5f457f2dd400eac882fb7d9b157cda6.png)

There are 3 shards that have very different sizes, looking closer I noticed the 3 of them were primary shards, and checking at the size of their respective replicas I saw there is a large difference:

```
| Shard# | Replica/Primary | Size |
| 5 | p |27.9gb|
| 5 | r |14.6gb|
| 6 | p |40.6gb|
| 6 | r |14.6gb|
| 7 | p |44.1gb|
| 7 | r |14.5gb|

```

I took a look at the document counts of every shard and saw they are all the same, except for these 3 shards, where there is small difference in documents (max 44 documents, and our documents are small, approx. 1 kilobyte each one). I'd like to know which documents are the different ones but I don't know how to do it (information is appreciated).

What can be the cause of this large difference?

**BTW:** I have read other threads about this topic ([here](https://discuss.elastic.co/t/number-of-docs-differents-between-primary-shards-and-replicas/76228/4) and [here](https://discuss.elastic.co/t/primary-and-replica-shard-is-very-different/137745)) so here is some more information in case it's useful:

- This index is not being currently heavily written (it was heavily written around 16 hours ago and then we stopped)
- We don't delete nor update documents, we only insert them

This is how I calculated the sizes and document counts:

```
$ curl -s 'mycluster/_cat/shards?h=i,s,p,d,sto | grep myindex'

myindex 0 p 24321381 14gb
myindex 0 r 24321381 14gb
myindex 1 p 24313216 13.9gb
myindex 1 r 24313216 13.9gb
myindex 2 p 24317009 14gb
myindex 2 r 24317009 13.9gb
myindex 3 p 24325100 14gb
myindex 3 r 24325100 13.9gb
myindex 4 p 24309484 13.9gb
myindex 4 r 24309484 13.9gb
myindex 5 p 24319283 27.9gb <- 
myindex 5 r 24319280 14.6gb <- 
myindex 6 p 24318975 40.6gb <- 
myindex 6 r 24318931 14.6gb <- 
myindex 7 p 24320967 44.1gb <- 
myindex 7 r 24320929 14.5gb <- 
myindex 8 p 24321325 13.9gb
myindex 8 r 24321325 14gb
myindex 9 p 24316102 13.9gb
myindex 9 r 24316102 13.9gb

```

And these are my index's settings:

```
$ curl -s 'mycluster/myindex/_settings' | jq .
{
  "myindex": {
    "settings": {
      "index": {
        "routing": {
          "allocation": {
            "include": {
              "data_type": "hot"
            }
          }
        },
        "search": {
          "slowlog": {
            "threshold": {
              "fetch": {
                <redacted>
              },
              "query": {
                <redacted>
              }
            }
          }
        },
        "indexing": {
          "slowlog": {
            "threshold": {
              "index": {
                <redacted>
              }
            }
          }
        },
        "number_of_shards": "10",
        "provided_name": "<redacted>",
        "max_result_window": "50000",
        "creation_date": "<redacted>",
        "analysis": {
          "normalizer": {
            <redacted>
          },
          "analyzer": {
            <redacted>
          }
        },
        "number_of_replicas": "1",
        "uuid": "<redacted>",
        "version": {
          "created": "6020499"
        }
      }
    }
  }
}

```

ElasticSearch Version:

```
$ curl -s 'mycluster/'
{
  "name" : "<redacted>',
  "cluster_name" : "<redacted>",
  "cluster_uuid" : "<redacted>",
  "version" : {
    "number" : "6.2.4",
    "build_hash" : "ccec39f",
    "build_date" : "2018-04-12T20:37:28.497551Z",
    "build_snapshot" : false,
    "lucene_version" : "7.2.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "<redacted>"
}
```

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 28, 2018, 8:45pm UTC](https://discuss.elastic.co/t/shard-size-is-different-between-primary-and-replica/150079/2 "2018-09-28T20:45:02Z")

</div>

It's possible you are affected by [https://github.com/elastic/elasticsearch/pull/30244](https://github.com/elastic/elasticsearch/pull/30244) which was fixed in 6.3.0. Can you upgrade to at least this version?

---

<div class="post-metadata">

**Author:** ![dwilches](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dwilches/32/48586_2.png) [@dwilches](https://discuss.elastic.co/u/dwilches)\
**Post date:** [September 28, 2018, 8:59pm UTC](https://discuss.elastic.co/t/shard-size-is-different-between-primary-and-replica/150079/3 "2018-09-28T20:59:15Z")

</div>

Thanks! Updating to ES 6.4.1 indeed solved the issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 26, 2018, 8:59pm UTC](https://discuss.elastic.co/t/shard-size-is-different-between-primary-and-replica/150079/4 "2018-10-26T20:59:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.