# Hot/Warm architecture and Index Stats

**URL:** <https://discuss.elastic.co/t/hot-warm-architecture-and-index-stats/182388>\
**Category:** Elasticsearch\
**Created:** [May 23, 2019, 9:28am UTC](https://discuss.elastic.co/t/hot-warm-architecture-and-index-stats/182388 "2019-05-23T09:28:54Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![prizov](https://avatars.discourse-cdn.com/v4/letter/p/958977/32.png) [@prizov](https://discuss.elastic.co/u/prizov)\
**Post date:** [May 23, 2019, 9:28am UTC](https://discuss.elastic.co/t/hot-warm-architecture-and-index-stats/182388/1 "2019-05-23T09:28:54Z")

</div>

Hi guys!

Could you explain why index in Elasticsearch loose stats after relocation in case of changing routing allocation rules?

Steps to reproduce:  
We have two zones hot and warm

1. Create index in hot zone  
`curl -XPUT localhost:9200/test-index`

2. Check that query total stats is equal 0  
`curl -s -XGET localhost:9200/test-index/_stats | jq '.indices["test-index"].primaries.search.query_total'`

3. Make some search request to test-index  
`curl -XGET localhost:9200/test-index/_search`

4. Check that query stats is changed  
`curl -s -XGET localhost:9200/test-index/_stats | jq '.indices["test-index"].primaries.search.query_total'`

5. Change allocation rule  
`curl -X PUT "localhost:9200/test-index/_settings" -H 'Content-Type: application/json' -d '{ "index" : { "routing.allocation.require": {"zone": "cold"} } }'`

6. Check stats -\> It's flushed  
`curl -s -XGET localhost:9200/test-index/_stats | jq '.indices["test-index"].primaries.search.query_total'` = 0

If there is some workaround to save index stats after relocation?  
Thanks!

---

<div class="post-metadata">

**Author:** ![luiz.santos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/luiz.santos/32/24664_2.png) [@luiz.santos](https://discuss.elastic.co/u/luiz.santos)\
**Post date:** [May 31, 2019, 1:39pm UTC](https://discuss.elastic.co/t/hot-warm-architecture-and-index-stats/182388/2 "2019-05-31T13:39:05Z")

</div>

Hi @prizov,

According to our [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html), it's an expected behavior:

> Note, as shards move around the cluster, their stats will be cleared as they are created on other nodes. On the other hand, even though a shard "left" a node, that node will still retain the stats that shard contributed to.

If you have [monitoring](https://www.elastic.co/guide/en/elasticsearch/reference/current/es-monitoring.html) enabled, the following query will show you the value along the time (per day) and also the derivative (difference) from the last day:

```auto
GET .monitoring-es-*/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": {
        "term": {
          "type": "index_stats"
        }
      }
    }
  },
  "aggregations": {
    "index_name": {
      "terms": {
        "field": "index_stats.index",
        "size": 100
      },
      "aggs": {
        "per_day": {
          "date_histogram": {
            "field": "timestamp",
            "interval": "day"
          },
          "aggs": {
            "searches": {
              "max": {
                "field": "index_stats.primaries.search.query_total"
              }
            },
            "search_derivative": {
              "derivative": {
                "buckets_path": "searches"
              }
            }
          }
        }
      }
    }
  }
}

```

Result:

```auto
...
  "aggregations" : {
    "index_name" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : ".kibana_1",
          "doc_count" : 24952,
          "per_day" : {
            "buckets" : [
              {
                "key_as_string" : "2019-05-28T00:00:00.000Z",
                "key" : 1559001600000,
                "doc_count" : 2790,
                "searches" : {
                  "value" : 2088.0
                }
              },
              {
                "key_as_string" : "2019-05-29T00:00:00.000Z",
                "key" : 1559088000000,
                "doc_count" : 8640,
                "searches" : {
                  "value" : 2944.0
                },
                "search_derivative" : {
                  "value" : 856.0
                }
              },
...

```

I hope it helps!

Cheers,  
Luiz Santos

---

<div class="post-metadata">

**Author:** ![prizov](https://avatars.discourse-cdn.com/v4/letter/p/958977/32.png) [@prizov](https://discuss.elastic.co/u/prizov)\
**Post date:** [June 4, 2019, 8:28am UTC](https://discuss.elastic.co/t/hot-warm-architecture-and-index-stats/182388/3 "2019-06-04T08:28:32Z")

</div>

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 2, 2019, 8:28am UTC](https://discuss.elastic.co/t/hot-warm-architecture-and-index-stats/182388/4 "2019-07-02T08:28:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.