Setting up a Separate Monitoring Cluster

Hi All,

I'm trying to setup a separate cluster(kibanacluster) for monitoring my primary elasticsearch cluster (marveltest) . Below are the ES, Marvel and Kibana versions I'm using. The ES version is fixed for the moment. I can update or downgrade the other components if needed.


The monitoring cluster and Kibana are both running in the host and the primary cluster is running in a separate host elasticsearch.yml has the below entry [""]

marvel.enabled: true

type: http
host: [""]

Looking at the DEBUG logs in the monitoring cluster i can see data is coming from the primary cluster but is getting "filtered" since the cluster name is different.

[2016-07-04 16:33:25,144][DEBUG][transport.netty ] [nodek] connected to node [{#zen_unicast_2#}{}{}]
[2016-07-04 16:33:25,144][DEBUG][transport.netty ] [nodek] connected to node [{#zen_unicast_1#}{}{}]
[2016-07-04 16:33:25,183][DEBUG][] [nodek] [1] filtering out response from {node1}{Rmgg0Mw1TSmIpytqfnFgFQ}{}{}, not same cluster_name [marveltest]
[2016-07-04 16:33:26,533][DEBUG][] [nodek] [1] filtering out response from {node1}{Rmgg0Mw1TSmIpytqfnFgFQ}{}{}, not same cluster_name [marveltest]
[2016-07-04 16:33:28,039][DEBUG][] [nodek] [1] filtering out response from {node1}{Rmgg0Mw1TSmIpytqfnFgFQ}{}{}, not same cluster_name [marveltest]
[2016-07-04 16:33:28,040][DEBUG][transport.netty ] [nodek] disconnecting from [{#zen_unicast_2#}{}{}] due to explicit disconnect call
[2016-07-04 16:33:28,040][DEBUG][discovery.zen ] [nodek] filtered ping responses: (filter_client[true], filter_data[false])
--> ping_response{node [{nodek}{vQ-Iq8dKSz26AJUX77Ncfw}{}{}], id[42], master [{nodek}{vQ-Iq8dKSz26AJUX77Ncfw}{}{}], hasJoinedOnce [true], cluster_name[kibanacluster]}
[2016-07-04 16:33:28,053][DEBUG][transport.netty ] [nodek] disconnecting from [{#zen_unicast_1#}{}{}] due to explicit disconnect call
[2016-07-04 16:33:28,057][DEBUG][transport.netty ] [nodek] connected to node [{nodek}{vQ-Iq8dKSz26AJUX77Ncfw}{}{}]
[2016-07-04 16:33:28,117][DEBUG][discovery.zen.publish ] [nodek] received full cluster state version 32 with size 5589

Thank you,

Please don't post pictures of text, they are difficult to read and some people may not be even able to see them.

Did you follow the docs, here -

Sorry about that. I have edited the post.
Yes I followed the above link in setting it up.
Could this be a licence issue? Since there are two clusters involved?

It's not a license issue.

Can you show the elasticsearch.yml file of each (but remove comments, and empty lines).

Hi Mark,

Below are the elasticsearch.yml files.

2.116 (Primary cluster) marveltest node1
bootstrap.mlockall: true ["", ""]
gateway.recover_after_nodes: 1
gateway.expected_nodes: 1
gateway.recover_after_time: 1m
path.repo: ["/usr/local/surf/ES_BACKUP"]
script.inline: true
script.indexed: true

2.124(Monitoring Cluster) kibanacluster nodek
node.master: false false ["", ""]

I'd suggest you read, as I mentioned.

However you don't point each cluster discovery at the other node.