# Aggregating and storing results for large indices **URL:** \ **Category:** Elasticsearch\ **Created:** [October 3, 2014, 1:36pm UTC](https://discuss.elastic.co/t/aggregating-and-storing-results-for-large-indices/20063 "2014-10-03T13:36:01Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Jason\_Motylinski](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_motylinski/32/1245_2.png) [@Jason\_Motylinski](https://discuss.elastic.co/u/Jason_Motylinski)\ **Post date:** [October 3, 2014, 1:36pm UTC](https://discuss.elastic.co/t/aggregating-and-storing-results-for-large-indices/20063/1 "2014-10-03T13:36:01Z")
We collect lots of log events from our web servers. Because of compute limitations we are only able to keep 4 hours worth of log data in Elasticsearch (~100mil documents for 4 hours). I'd like to run some high level aggregation queries every 10 minutes and store the results in an aggregated index. I was going to hack together a python script and throw a cron job out there to accomplish this but it seems like rivers would be a good solution for this as well. Is there an Elasticsearch river for ..Elasticsearch? Is there a better way to run a query on an interval and store the results? Thanks for the help, j. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d69d0205-4e8c-48ee-852d-a5c1016006a0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d69d0205-4e8c-48ee-852d-a5c1016006a0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jorge\_Luis\_Betancour](https://avatars.discourse-cdn.com/v4/letter/j/22d042/32.png) [@Jorge\_Luis\_Betancour](https://discuss.elastic.co/u/Jorge_Luis_Betancour)\ **Post date:** [October 4, 2014, 12:21am UTC](https://discuss.elastic.co/t/aggregating-and-storing-results-for-large-indices/20063/2 "2014-10-04T00:21:43Z")
I believe that rivers are in a path to be deprecated (not right now but in the future) so I'll recommend you to write your own logic outside of ES. Regards, On Oct 3, 2014, at 9:36 AM, Jason Motylinski [jason@motylinski.com](mailto:jason@motylinski.com) wrote: > We collect lots of log events from our web servers. Because of compute limitations we are only able to keep 4 hours worth of log data in Elasticsearch (~100mil documents for 4 hours). > > I'd like to run some high level aggregation queries every 10 minutes and store the results in an aggregated index. I was going to hack together a python script and throw a cron job out there to accomplish this but it seems like rivers would be a good solution for this as well. Is there an Elasticsearch river for ..Elasticsearch? Is there a better way to run a query on an interval and store the results? > > Thanks for the help, > > j. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d69d0205-4e8c-48ee-852d-a5c1016006a0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d69d0205-4e8c-48ee-852d-a5c1016006a0%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). Concurso "Mi selfie por los 5". Detalles en [http://justiciaparaloscinco.wordpress.com](http://justiciaparaloscinco.wordpress.com) -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/58F3F1FF-38E2-467D-9588-610D729D692E%40uci.cu](https://groups.google.com/d/msgid/elasticsearch/58F3F1FF-38E2-467D-9588-610D729D692E%40uci.cu). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:58am UTC](https://discuss.elastic.co/t/aggregating-and-storing-results-for-large-indices/20063/3 "2017-07-06T00:58:18Z")