Query Metrics and Index to ElasticSearch

Elastic Stack Elasticsearch
1

I am using File Beat & LogStash to Collect the logs from various source and indexing the raw logs/CDRs to ES 5.0 and the key purpose of monitoring is the transaction KPIs such as TPS, Latency min wise or so... The aggregation/summary of the KPIs is done by Kibana over detailed logs.
I would need to generate the aggregated KPI historical statistics for reporting purpose in Kibana..

Is there anyway I can run the Queries in ES to collect the aggregated KPIs and store/redirect the output into a KPI Index.

Appreciate if any one can give sample example.

Many Thanks
Venkatesh

2

You mean you want to periodically calculate some statistics and then store them back into another index? I don't think there's a dedicated API for that, people usually use a client to query one index, potentially manipulate the results in some ways and store that somewhere else (e.g. another index).

3

Hi
Yes. Say i want to query from the "logs index" at the mid night for metrics like transaction count per min ..etc and store into another index.

Is there any sample client program/script to guide me on how to do this.