Monitoring production cluster with Kibana and Metricbeat

Hi there,

we run an Elastic Stack version 6.8.0 with a basic licence.

Like a few others we have issues monitoring the production cluster in Kibana.
As recommended we have set up a separate monitoring cluster including a separate Kibana instance. All monitoring data from the production cluster is shipped via Metricbeat. Thus there are some metricbeat-* indices in Elasticsearch (monitoring).
Hitting the Monitoring section in Kibana (monitoring) is says " Monitoring is currently off" and we need to turn it on. Doing so monitors the monitoring cluster itself.

We have seen this solution but they use HTTP exporters which is undesired.

Some maybe relevant Kibana settings:

elasticsearch.hosts: <monitoring cluster> (if we change this to <production cluster> it says "No monitoring data found", which makes totally sense, right?)
xpack.monitoring.kibana.collection.enabled: false

And some Elasticsearch settings:

xpack.monitoring.collection.enabled: false

What are we missing?
Regards

You will need to specify the information on the monitoring cluster in your kibana.yml of the Kibana instance which is part of your primary cluster.

https://www.elastic.co/guide/en/kibana/6.8/monitoring-settings-kb.html

xpack.monitoring.elasticsearch.*

Hello and thanks for the reply.

We want to use a separate Kibana instance to monitor the production cluster including the production Kibana instance. If I get your suggestion right, we would use the production Kibana instance for monitoring, but that is undesired.

Hi again,

even if I configure the production Kibana instance so that xpack.monitoring.elasticsearch.hosts points to the monitoring cluster, no monitoring visualizations show up. It always says "We couldn't activate monitoring. No monitoring data found.". The monitoring cluster (single node actually) has an index named metricbeat-6.8.4- 2019.11.12 filled with metrics received from production cluster. Why doesn't Kibana read from that index? What setting is missing?

Thank you