I am Currently using File Beat & LogStash to Collect the logs from various source and indexing the raw logs/CDRs to ES and the key purpose of monitoring is the transaction KPIs such as TPS, Latency min wise or so... The aggregation/summary of the KPIs is done by Kibana over detailed logs.
I have got space constraints and can't keep the huge volume log indexes for long. However I would still need the aggregated KPI historical statistics for reporting purpose..
Is there anyway I can run the Queries in ES to collect the aggregated KPIs and store/redirect the output into a KPI Index.
I have seen some threads that it is technically possible .. But I have not got any clear idea on how to do it.. Appreciate if any one can give more details / specifics/ examples..