I have established a separate monitoring cluster based on recommended best practices (current ES version 6.5.3). Initially, I followed the instructions listed at https://www.elastic.co/guide/en/kibana/7.0/monitoring-metricbeat.html; however the only monitoring data that appeared in the monitoring cluster was its own.
Frustrated, I did some more research and was pleased to find another user who had encountered the exact same scenario: Independant monitoring cluster.
I followed the recommendations of the user in that post, and was happy to see my production data appear in the monitoring cluster. However; much to my dismay the data still appeared in the production cluster as well (at least the elasticsearch portion). Doesn't this pretty much defeat the purpose?
So, instead of configuring the http forwarders for the monitoring cluster and then routing some of the data from beats to PROD, I routed everything to the ES Monitoring instance. This time, I was able to see everything in the monitoring cluster and nothing in PROD. I made the change hours ago and everything is working as intended; EXCEPT now when I click on monitoring in PROD I get a warning message suggesting I may have a misconfiguration (attached). I do not have a misconfiguration. This solution is working exactly as I intended, and now I have to explain all this to my client. Can I get rid of this warning message, and if not; is there a VERIFIED way of making this work otherwise? I'm really hoping this type of warning message in the UI isn't just a thinly veiled attempt at forcing us to buy the license... Please advise what config files you need and
I will gladly share.
Hi Sam,
If you do not want the monitoring UI, you can easily disable it in the kibana.yml.
I have disabled a few of the apps I do not use in my production environment. Keeps the UI clean.
xpack.monitoring.ui.enabled: false
xpack.apm.enabled: false
Hope this helps!
Thanks Nachiket! Also kudos for figuring out the solution to the monitoring issue! Very nice!
That removes the error; but is there a way to suppress the 404 message I now get for monitoring when kibana starts?
Thanks sam.
What does the error look like? I do not see any errors in my environment.
The error is gone; seemed like a one time thing. Quick question though; when you set up HTTP forwarders; did you do so in Elasticsearch.yml (main config file for Elasticsearch), or did you put them in the elasticsearch.yml module for Metricbeat? Also, is there a limit that you know of for how many beats you can monitor with free license?
HTTP Forwarder are set in the elasticsearch.yml for Elasticsearch, and not in metricbeat.
No, there isn't as far as I know. But haven't tested using more than 50 at a a time.
So, this is working now (thank you Nachiket). However, one of my nodes that has beats installed is not showing up in the monitoring. The only difference between this and the other nodes is elasticsearch is not installed. Do you have to install elasticsearch on all the nodes that utilize http forwarders? I have the beats on this node pointed at my elastic instances.
I am not quite sure what you mean by this.
I currently have beats installed on all my production machines and the xpack.monitoring.elasticsearch setting currently points to my Production Elasticsearch.
On all my Production Elasticsearch nodes, I have set the http forwarder to send the data to my monitoring cluster. There is no need to run Elasticsearch on every node where you run metricbeat.
Thank you. This issue is resolved.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.