# Rollup - Store statistics (just a number)

**URL:** https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214
**Category:** Elasticsearch
**Created:** [May 12, 2020, 11:13am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214 "2020-05-12T11:13:54Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)
#### Post date: [May 12, 2020, 11:13am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/1 "2020-05-12T11:13:54Z")

</div>

Hi.

I need to store historical data from my indexes, exactly the number of request per minute that my webservices has along time. Its possible to store only this information? If i create a rollup job using only the fileds i need to extract this information, the index size becomes so big depending on the days that the job process. I only want numerical statistics. Its possible to do that?

Thanks in advance

Víctor

---

<div class="post-metadata">

### Author: ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [May 12, 2020, 12:58pm UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/2 "2020-05-12T12:58:20Z")

</div>

Rollup stores extra meta information in order to provide rollup search, if you do not need rollup search, [transform](https://www.elastic.co/guide/en/elasticsearch/reference/current/transforms.html) might be an option for you, it only stores what you are asking for and in case you want to further compress, you can tweak the mappings to use smaller data types.

Whether rollup or transform, the reduction should mainly depend on the bucket size you choose, in your case the date histogram interval.

---

<div class="post-metadata">

### Author: ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)
#### Post date: [May 13, 2020, 9:40am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/3 "2020-05-13T09:40:47Z")

</div>

Hi Hendrik.

Transform is what i need! I created a transform to extract only the information i need, but when i go to Discover and use the new index i created, theres no option to filter by time. What can i do to filter by time?

Kind regards

Víctor

---

<div class="post-metadata">

### Author: ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [May 13, 2020, 9:59am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/4 "2020-05-13T09:59:58Z")

</div>

Hi,

that's a current limitation, see this [issue](https://github.com/elastic/kibana/issues/59531). The workaround is to either create the index pattern yourself, not using the transform wizard or you delete the already existing index pattern and create a new one. The limitation in the management UI has its own [issue](https://github.com/elastic/kibana/issues/9212) (contains a 3rd option: manually update the index pattern).

Hope that fixes it.

---

<div class="post-metadata">

### Author: ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)
#### Post date: [May 13, 2020, 11:36am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/5 "2020-05-13T11:36:53Z")

</div>

Got it!

Thanks a lot Hendrik

---

<div class="post-metadata">

### Author: ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)
#### Post date: [May 14, 2020, 7:25am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/6 "2020-05-14T07:25:12Z")

</div>

Hi Hendrik.

One more question. If i want to filter by time in Discover, Its required in the transform to group index data by Timestamp? Is there another way to do it?

Regards

![Captura](https://us1.discourse-cdn.com/elastic/original/3X/6/0/60687b0566e529092984ee35707692f64fa0c997.png)

---

<div class="post-metadata">

### Author: ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [May 14, 2020, 8:28am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/7 "2020-05-14T08:28:10Z")

</div>

I think in the `group_by` it makes the most sense. You can of course have time fields in aggregations, too. E.g. a `last_updated` field. Still if you do not `group_by time`, it will not result in a time series.

I am not sure I am getting your question, can you explain what you want to do?

---

<div class="post-metadata">

### Author: ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)
#### Post date: [May 14, 2020, 9:02am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/8 "2020-05-14T09:02:57Z")

</div>

I´m sending all traffic from my F5 load balancer to Elastic. At the moment just for 5 services, wich are indentified by the virtual\_ip field. For each service, there are to many diferent requests, wich i differentiate by the http\_path field. All i want to do is a transform to store in an index, total request for an specific virtual\_server, total requests for an specific http\_path of that virtual\_server, average response time and also if its possible, status of each request. Heres my transform code:

```
POST _transform/_preview
{
 "source": {
"index": "my_index",
"query": {
  "bool": {
"must": [
  {
    "match": {
      "virtual_ip": "x.x.x.x"
    }
  }
],
"filter": [
  {
    "term": {
      "http_path.keyword": "my_path/my_file.aspx"
    }
  },
  {
    "range": {
      "@timestamp": {
        "time_zone": "+02:00",
        "gte": "2020-05-07T00:00:00",
            "lte": "2020-05-08T00:00:00"
              }
            }
          }
        ]
}
}
 },
 "dest": {
   "index": "my_dest_index"
},
"pivot": {
"group_by": {
  "status.keyword": {
    "terms": {
      "field": "status.keyword"
    }
  }
},
"aggregations": {
  "response_msecs.avg": {
    "avg": {
      "field": "response_msecs"
    }
  },
  "count": { "value_count": { "field": "@timestamp" }}
 }
 }
}

```

I dont want to use time range because that means i will have to create an index for each day for example.

Regards

---

<div class="post-metadata">

### Author: ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)
#### Post date: [May 14, 2020, 9:35am UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/9 "2020-05-14T09:35:41Z")

</div>

Thanks, always easier to work with examples.

Adding a `date_histogram` in `group_by` should work, however you said you want "another way". What's the problem with something like this:

```auto
"group_by": {
  "day_bucket": {
    "date_histogram": {
      "field" : "@timestamp",
      "calendar_interval" : "1d"
    }
  },
  "status.keyword": {
    "terms": {
      "field": "status.keyword"
    }
  }
}

```

---

<div class="post-metadata">

### Author: ![vdelcampo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vdelcampo/32/65731_2.png) [@vdelcampo](https://discuss.elastic.co/u/vdelcampo)
#### Post date: [May 14, 2020, 12:48pm UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/10 "2020-05-14T12:48:56Z")

</div>

That is what i was looking for! Thank you very much Hendrik!

Kind Regards

Víctor

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 11, 2020, 12:49pm UTC](https://discuss.elastic.co/t/rollup-store-statistics-just-a-number/232214/11 "2020-06-11T12:49:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
