Monitor the indexing rate of specific index?

Elastic Stack Elasticsearch
1

We are using fluentd to send logs to ElasticSearch:

<store>
    @type elasticsearch
    host elasticsearch
    port 9200
    logstash_format true
    index_name logstash
    type_name logstash
</store>

For some reasons, sometimes index is not being created, and Kibana has nothing to display in Last 7 days.

I would like to monitor this situation to send me an alert whenever it happens. In Kibana, we have a graph that displays the number of messages per 30 seconds (for logstash-* index). What is the corresponding metric in ElasticSearch: https://github.com/influxdata/telegraf/tree/master/plugins/inputs/elasticsearch

so I can add metrics and setup an alert for it in Grafana?

2

Check the .monitoring-es-* indices. All monitoring data to into those indices.

3

Only if you have X-Pack with Monitoring enabled.

I don't believe there is a specific API response that provides this. The Monitoring functionality extracts it from the existing API responses.

1 Like
4

Check the .monitoring-es-* indices. All monitoring data to into those indices.

What can I do with those indices?

Only if you have X-Pack with Monitoring enabled.

After enabling X-Pack monitoring feature, I can see that there is a graph for Indexing rate with an explanation for Primary shards is:

The number of documents are being indexed for the primary shards

Question: There is no corresponding field in Cluster stats?Cluster stats API | Elasticsearch Guide [8.11] | Elastic

5

Those indices contains the raw data that feeds the Monitoring graphs related to Indexing Rate

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.