The issue is resolved. For the sake of others who would like to do this, here are the settings I made:
My production cluster has the following cluster settings:
{
"persistent" : {
"xpack" : {
"monitoring" : {
"elasticsearch" : {
"collection" : {
"enabled" : "true"
}
},
"collection" : {
"enabled" : "true"
}
}
}
},
"transient" : { }
}
My monitoring cluster has the following cluster settings:
{
"persistent" : {
"xpack" : {
"monitoring" : {
"elasticsearch" : {
"collection" : {
"enabled" : "false"
}
},
"collection" : {
"enabled" : "true"
}
}
}
},
"transient" : { }
}
And I have configure http exporters in elasticsearch.yml to point to my Monitoring cluster.
I have then configured beats and logstash to send the monitoring data to my Production cluster, and not the Monitoring cluster.
Also, I have disabled kibana monitoring the kibana.yml
.
xpack.monitoring.kibana.collection.enabled: false
Post this and deleting all the existing .monitoring*
indices on my Monitoring cluster, I was able to view the monitoring data in the Kibana UI.