My monitoring Indexes are very large, what are best practices?

[Jbrar (Jatinder)]

I have a similar issue as to what was raised in How to change indexing frequency of X-Pack monitoring plugin? that my .monitoring indexes are disproportionately large in comparison to my created indexes on Elastic.

In looking at the data that is saved with regards to the index_stats,index_recovery,Indices_stats, node_stats and other mappings nothing seems of note. In looking at xpack.monitoring.collection.interval and xpack.monitoring.history.duration settings, I can make setting changes to increase the interval length and increase the frequency with how often data is being cleaned up. However, I don't know if that solves the problem.

At the end of the day if I increase xpack.monitoring.collection.interval I would be reducing how often monitoring checks up on my cluster, right? If the goal is to maintain the same monitoring frequency but only update the .monitoring collection when something is spotted that is noteworthy (ie. failures, slow performance etc.), what configuration change would I need to make to kibana.yml and elastic.yml?

[warkolm (Mark Walkom)]

@Jbrar

At the end of the day if I increase xpack.monitoring.collection.interval I would be reducing how often monitoring checks up on my cluster, right?

Yes.

@Jbrar

If the goal is to maintain the same monitoring frequency but only update the .monitoring collection when something is spotted that is noteworthy (ie. failures, slow performance etc.), what configuration change would I need to make to kibana.yml and elastic.yml?

That's not currently possible, it's something we'd like to do have in future.

[Jbrar (Jatinder)]

Fair enough. Thanks so much for the guidance. Much appreciated.