We collect lots of log events from our web servers. Because of compute
limitations we are only able to keep 4 hours worth of log data in
Elasticsearch (~100mil documents for 4 hours).
I'd like to run some high level aggregation queries every 10 minutes and
store the results in an aggregated index. I was going to hack together a
python script and throw a cron job out there to accomplish this but it
seems like rivers would be a good solution for this as well. Is there an
Elasticsearch river for ..Elasticsearch? Is there a better way to run a
query on an interval and store the results?
Thanks for the help,
j.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d69d0205-4e8c-48ee-852d-a5c1016006a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
I believe that rivers are in a path to be deprecated (not right now but in the future) so I'll recommend you to write your own logic outside of ES.
Regards,
On Oct 3, 2014, at 9:36 AM, Jason Motylinski jason@motylinski.com wrote:
We collect lots of log events from our web servers. Because of compute limitations we are only able to keep 4 hours worth of log data in Elasticsearch (~100mil documents for 4 hours).
I'd like to run some high level aggregation queries every 10 minutes and store the results in an aggregated index. I was going to hack together a python script and throw a cron job out there to accomplish this but it seems like rivers would be a good solution for this as well. Is there an Elasticsearch river for ..Elasticsearch? Is there a better way to run a query on an interval and store the results?
Thanks for the help,
j.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d69d0205-4e8c-48ee-852d-a5c1016006a0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Concurso "Mi selfie por los 5". Detalles en http://justiciaparaloscinco.wordpress.com
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/58F3F1FF-38E2-467D-9588-610D729D692E%40uci.cu.
For more options, visit https://groups.google.com/d/optout.