# Storing aggregation results back into elasticsearch
**URL:** https://discuss.elastic.co/t/storing-aggregation-results-back-into-elasticsearch/17932
**Category:** Elasticsearch
**Created:** [June 5, 2014, 4:51pm UTC](https://discuss.elastic.co/t/storing-aggregation-results-back-into-elasticsearch/17932 "2014-06-05T16:51:52Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@erewh0n](https://discuss.elastic.co/u/erewh0n)
#### Post date: [June 5, 2014, 4:51pm UTC](https://discuss.elastic.co/t/storing-aggregation-results-back-into-elasticsearch/17932/1 "2014-06-05T16:51:52Z")
I've recently started using and enjoying ES, in particular I'm keen to
exploit the new aggregations feature to report on system metrics data that
is currently being fed into ES indexes.
I'm experimenting with aggregations that fold up things like request rates
per machine or API calls (per machine, globally, etc). I was thinking that
it might be useful to store the aggregation result itself, particularly if
I set a (let's say) weekly TTL on the incoming metrics data but would like
to preserve historical aggregates (e.g. find me the average/min/max request
rate on day 17). I might want to keep the raw metrics for a week, but the
aggregates should potentially stick around for years.
Are there any recommendations/patterns in regards to dealing with these
scenarios? Are there existing means for recomputing aggregates at regular
intervals and emitting those back into ES?
Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4e9acaaf-4702-4d50-a16a-e375efe431e1%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4e9acaaf-4702-4d50-a16a-e375efe431e1%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@erewh0n](https://discuss.elastic.co/u/erewh0n)
#### Post date: [June 5, 2014, 10:19pm UTC](https://discuss.elastic.co/t/storing-aggregation-results-back-into-elasticsearch/17932/2 "2014-06-05T22:19:29Z")
To clarify, these questions are coming from my desire to dynamically
produce real time aggregated information from a "stream", which in this
case is metric data we're feeding to ES. I'm concerned about unnecessary
re-execution of aggregations on (potentially large) data sets that could be
computed more efficiently by maintaining buckets that are simply updated as
data enters ES. I'm not sure if there is a good pattern for this or if I'm
better off using a different technology entirely (e.g. Storm, etc), though
it is nice having all my logs/metrics queryable from one place.
On Thursday, June 5, 2014 12:51:52 PM UTC-4, erewh0n wrote:
> I've recently started using and enjoying ES, in particular I'm keen to
> exploit the new aggregations feature to report on system metrics data that
> is currently being fed into ES indexes.
>
> I'm experimenting with aggregations that fold up things like request rates
> per machine or API calls (per machine, globally, etc). I was thinking that
> it might be useful to store the aggregation result itself, particularly if
> I set a (let's say) weekly TTL on the incoming metrics data but would like
> to preserve historical aggregates (e.g. find me the average/min/max request
> rate on day 17). I might want to keep the raw metrics for a week, but the
> aggregates should potentially stick around for years.
>
> Are there any recommendations/patterns in regards to dealing with these
> scenarios? Are there existing means for recomputing aggregates at regular
> intervals and emitting those back into ES?
>
> Thanks!
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/61b81fce-373b-4a59-bcfa-d5cacb7b5744%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/61b81fce-373b-4a59-bcfa-d5cacb7b5744%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:24am UTC](https://discuss.elastic.co/t/storing-aggregation-results-back-into-elasticsearch/17932/3 "2017-07-06T01:24:22Z")