Monitoring could not connect to elasticsearch, failed with xpack monitoring not available

[jasl (JoseAlberto)]

So I have 2 ES clusters:

• Main:

  • 6.2
  • xpack
  • monitoring enabled
  • filebeat

• Monit

  • 6.2
  • xpack
  • monitoring disabled
  • kibana with monitoring enabled

I can see monitoring data from Main in Monit's kibana (coming from ES itself), but in filebeat I get this error:

Monitoring could not connect to elasticsearch, failed with xpack monitoring not available

Seems like filebeat xpack.monitoring.enabled needs monitoring enabled in Monit cluster too. Is this a bug or a miss-configuration?

[rashmi (kulkarni)]

Hi

Can you please share your Filebeat config as well. It will be easier to debug . Just the ES output sections (if applicable) and the xpack.monitoring section should be enough to further probe. Also is your xpack.monitoring.elasticsearch.url pointing to the monitoring cluster, but monitoring is disabled in that cluster. If so, it won't have the special bulk endpoint ?
I talked with our developers on this issue, and they opine that in 7.0 it will be so much easier.

Cheers
Rashmi

[jasl (JoseAlberto)]

I understand the issue now. Monitoring configuration in filebeat refers to monitoring filebeat itself, but it cannot do it if the Monit cluster has not monitoring enabled.

I guess this is a intentional decision, but this behaviour is different from elasticsearch itself (as you can save ES monitoring data in a cluster with no monitoring enabled)