For storage purposes, I have to convert minutely data to hourly data, store it as an index. Likewise, after a week, convert the hourly data to weekly data. Is it possible to do it with ELK?
Technically it's possible, although we don't have any automated features to do it right now.
Essentially, you'll have to setup a cron job (or similar) that periodically executes an aggregation to collected the rollup summaries, then index those back into Elasticsearch. You could potentially use Watcher to do this instead of a cron-job, but the process is essentially the same.
After indexing the summaries you could do something like a Delete-by-query to remove the old documents, or perhaps just delete the source index when done.
We'd been exploring how to do automatic rollups internally in Elasticsearch, but right now it's all just internal experimentation. Someday maybe! 
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.