I have a separate monitoring cluster separate to the production one. However, when the monitoring cluster does not respond (e.g. maintenance, network problem, internal error) the production cluster goes RED and stop working. The kibana connected to the production get 500 errors.
Is there a way to get have a less disruptive behavior by the production cluster if monitoring is unreachable? Am I doing something wrong?
In each elasticsearch production node, I have this settings for xpack monitoring:
xpack.monitoring.enabled: true xpack.monitoring.exporters: id1: type: http host: ["elasticsearch_domain"] auth.username: "monitoring" auth.password: "monitoring_password"