# Create monthly index for metricbeat daily data

**URL:** <https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096>\
**Category:** Elasticsearch\
**Created:** [September 20, 2017, 4:05am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096 "2017-09-20T04:05:13Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![pasifus](https://avatars.discourse-cdn.com/v4/letter/p/59ef9b/32.png) [@pasifus](https://discuss.elastic.co/u/pasifus)\
**Post date:** [September 20, 2017, 4:05am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/1 "2017-09-20T04:05:13Z")

</div>

Hi

My cluster get data from metricbeat and save daily elasticsearch indexes. I'm intresting in best way to convert daily to monthly indexes. The goal is to decreas index storage size and show monthly view of performance.

I know that to easy way to create monthly indexes it to use reindex but I'm didn't found some example for process some fields before create monthly index. (to example: daily average for system.cpu.idle.pct, system.cpu.system.pct etc. fields). May it have another way using curator?

Thanks

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 20, 2017, 5:15am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/2 "2017-09-20T05:15:29Z")

</div>

You can do that in metricbeat config - [https://www.elastic.co/guide/en/beats/metricbeat/current/elasticsearch-output.html#\_index](https://www.elastic.co/guide/en/beats/metricbeat/current/elasticsearch-output.html#_index)

---

<div class="post-metadata">

**Author:** ![pasifus](https://avatars.discourse-cdn.com/v4/letter/p/59ef9b/32.png) [@pasifus](https://discuss.elastic.co/u/pasifus)\
**Post date:** [September 20, 2017, 5:22am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/3 "2017-09-20T05:22:58Z")

</div>

How exactly changes in indexes name helps me to convert daily metrics to monthly?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 20, 2017, 7:20am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/4 "2017-09-20T07:20:15Z")

</div>

It doesn't, it stores data in monthly indices as daily points.

Are you saying you want to store monthly values for things like CPU use?

---

<div class="post-metadata">

**Author:** ![pasifus](https://avatars.discourse-cdn.com/v4/letter/p/59ef9b/32.png) [@pasifus](https://discuss.elastic.co/u/pasifus)\
**Post date:** [September 20, 2017, 9:47am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/5 "2017-09-20T09:47:09Z")

</div>

I want extra index with monthly metrics than include daily average metrics.

// monthly\_filed != daily\_filed\*30  
for i range(1,30)  
monthly\_field[i] = avg(daily\_fileds[i])

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 20, 2017, 9:48am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/6 "2017-09-20T09:48:12Z")

</div>

Why not just use the aggregation APIs to do it for you?

---

<div class="post-metadata">

**Author:** ![pasifus](https://avatars.discourse-cdn.com/v4/letter/p/59ef9b/32.png) [@pasifus](https://discuss.elastic.co/u/pasifus)\
**Post date:** [September 20, 2017, 10:01am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/7 "2017-09-20T10:01:37Z")

</div>

> [@warkolm](#):
>
> aggregation API

How I say befe I use metricbeat. My metricbeat include ~600 fields. To create aggregation for all fields it's a long time. I'm looking for best practices. It's looks like many people uses this feature.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 20, 2017, 10:03am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/8 "2017-09-20T10:03:49Z")

</div>

It should be only a matter of seconds at the most for Elasticsearch to calculate this.

---

<div class="post-metadata">

**Author:** ![pasifus](https://avatars.discourse-cdn.com/v4/letter/p/59ef9b/32.png) [@pasifus](https://discuss.elastic.co/u/pasifus)\
**Post date:** [September 20, 2017, 10:18am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/9 "2017-09-20T10:18:22Z")

</div>

> [@warkolm](#):
>
> It should be only a matter of seconds at the most for Elasticsearch to calculate this.

🙂  
I mean it's long time work for me to add all fields.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 18, 2017, 10:19am UTC](https://discuss.elastic.co/t/create-monthly-index-for-metricbeat-daily-data/101096/10 "2017-10-18T10:19:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
