Elastic cluster metrics

Elastic Stack Elasticsearch
1

I want to create the dashboards in my application showing the elastic cluster data and metrics as it is shown in Kibana app monitoring. For getting most of the data I can use cluster health apis. But I not getting much on how to get data over some time period. Like if I want to show search rate or indexing rate for last 7 days, how to get all this data?

Thanks in advance.

2

I think you can use Metricbeat to bring your cluster metrics into a dashbaord
Check it here

3

What sort of app are you wanting to measure this from?

4

I have java spring back-end, where i want to get the data and process it.

5

And you want the query rates that it makes against Elasticsearch? Or something else.

6

Hi Warkolm, I have a elasticsearch setup in which i am storing the various logs by creating various indices. Now i want to create dashboards which shows the various details like docs count, docs size, indexing rate, searching rate for a particular index. So for getting this data we have apis like GET /<indexname>/_stats but the problem is it gives only current data. I want to show these data for any time period.

7

Hi ylasri, yeah metricbeat can help with this, but i am not getting all data from metricbeat like in GET /<indexname>/_stats we get the data like indexing.index_total, indexing.index_time_in_millis, etc. So in metricbeat i am not getting such kind of data.

8

Why not just use the Monitoring functionality we provide?

9

I want to provide such functionality in my application. I am building logging system for some users so decided to have this monitoring functionality as well on my app also.

10

If the x-pack monitoring gets the data, you can get it from the .monitoring* indexes but they are huge in terms of the number of fields (and nesting), though mostly self-documenting.

We are digging into these also for our product - basically you pick the metrics you want and pull them for the time period you want, but watch the scope as some are cluster level, some node level, etc. as I recall (I've not dug too deep yet; maybe will blog on it soon).

11

Thanks for your help, .monitoring-es-* indices contains all the info I needed.

1 Like
12

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.