# Rollup of apm indices

**URL:** <https://discuss.elastic.co/t/rollup-of-apm-indices/236327>\
**Category:** APM\
**Created:** [June 9, 2020, 12:09pm UTC](https://discuss.elastic.co/t/rollup-of-apm-indices/236327 "2020-06-09T12:09:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [June 9, 2020, 12:09pm UTC](https://discuss.elastic.co/t/rollup-of-apm-indices/236327/1 "2020-06-09T12:09:39Z")

</div>

Hello!

Has anyone tried to rollup APM indices? Is it even possible?

Thank you in advance for any feedback 🙂

---

<div class="post-metadata">

**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\
**Post date:** [June 9, 2020, 1:08pm UTC](https://discuss.elastic.co/t/rollup-of-apm-indices/236327/2 "2020-06-09T13:08:29Z")

</div>

Hello @A_B,  
can I ask about your use case for rollups with APM?

In case you are only looking for a way to manage data retention, you can make use of the index lifecycle management, which is enabled by default. This allows to keep more fine granular data (e.g. spans) or data that are outdated faster (e.g. errors) for a shorter period than high level data (e.g. transactions). APM Server sets up indices with the same retention period by default, but the index lifecycle policies are configurable. Read more about [how to configure ILM policies](https://www.elastic.co/guide/en/apm/server/current/ilm.html#custom-ilm-policy) for your data .

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [June 9, 2020, 1:20pm UTC](https://discuss.elastic.co/t/rollup-of-apm-indices/236327/3 "2020-06-09T13:20:24Z")

</div>

Hi @simitt,

thanks for the suggestions. Ideally, I would like to drop _resolution_ after some time. We e.g definitely do not need every trace after say two weeks but it would be nice to keep a few samples. The traces seem to be the ones using most storage. I know you can sample tracing in the agent. We would like to not have to do that...

---

<div class="post-metadata">

**Author:** ![simitt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simitt/32/106406_2.png) [@simitt](https://discuss.elastic.co/u/simitt)\
**Post date:** [June 10, 2020, 2:43pm UTC](https://discuss.elastic.co/t/rollup-of-apm-indices/236327/4 "2020-06-10T14:43:16Z")

</div>

A `trace` is actually a logical concept containing `transaction` and `span` documents, and potentially related `error` documents. All of these documents are stored in separated indices. Therefore if you would like to keep a sample of `traces`, one would need to figure out which root transactions (not having a `parent_id`) to keep and only delete unrelated documents from the other indices.

Unfortunately there is no out-of-the box logic at the moment taking care of this. As you already pointed out, the recommendation would be to enable sampling already in the agents.

---

<div class="post-metadata">

**Author:** ![A\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_b/32/17104_2.png) [@A\_B](https://discuss.elastic.co/u/A_B)\
**Post date:** [June 10, 2020, 2:51pm UTC](https://discuss.elastic.co/t/rollup-of-apm-indices/236327/5 "2020-06-10T14:51:42Z")

</div>

Right, that is true. Thanks for pointing that out @simitt

I was toying with the idea of using some sort of conditional document routing based on maybe `transaction.id` to different indices. Something like all `transaction.id` s ending in a certain sequence, go to a _special_ index, which will not be deleted by the ILM policy.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 1, 2020, 10:51am UTC](https://discuss.elastic.co/t/rollup-of-apm-indices/236327/6 "2020-07-01T10:51:45Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
