Stack Monitoring only viewing current index

Elastic Observability Metrics
1

Hi,

I am using ILM with metricbeat, within stack monitoring it seems i cannot view data beyond the current index, the previous indexs are available. My ILM policy is simply hot, 30 days or 50GB, no move to warm.

I am on 7.6.1

Thanks
Phil

2

Hi @probson,

Stack Monitoring indices (.monitoring-*) are not curated or managed by any ILM policy. Curation happens manually by Elasticsearch through these settings. Any ILM policies created by Metricbeat do not apply to monitoring indices.

Does that help?

3

Hi,

I should say that in using metric beat with the xpack modules to monitor rather than xpack monitoring.

Kind regards
Phil

4

Hi @probson,

Unfortunately, that doesn't matter for this situation.

The elasticsearch Metricbeat module indexes into different indices based on the xpack.enabled flag (which is enabled by default with the elasticsearch-xpack module). If xpack.enabled is true, Metricbeat will index into .monitoring-es-* indices. If xpack.enabled is false (or omitted), Metricbeat will index into metricbeat-* indices (or a custom index pattern if one is defined in metricbeat.yml).

The ILM policy that applies to metricbeat-* (or a custom index pattern defined in metricbeat.yml) does not have any effect on .monitoring-es-* indices.

It's confusing because Metricbeat almost always indexes into a consistent index pattern, but the xpack.enabled flag for the Elastic Stack modules will change this behavior.

Does that help?

5

Hi @chrisronline,

Checked my config, it was logstash that i had xpack.monitoring.enabled: false, although can see .monitoring-logstash indices. Using metricbeat and xpack module for all servers.

I guess in elasticsearch.yml i need to set xpack.monitoring.history.duration
What would be curating the monitoring.logstash indices in this case?

Thanks
Phil

6

HI @probson,

xpack.monitoring.history.duration setting in Elasticsearch covers curation for all monitoring documents that live in that cluster, not just .monitoring-es-* documents.

7

Thanks @chrisronline, sounds like that is what i was looking for.

Kind regards
Phil

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.