We set up a 3 nodes elasticsearch cluster, 1 Master (data) node, 1 data node, and 1 client node.
Client node elasticsearch.yml file
cluster.name: elkstats
node.name: clientnode
node.master: false
node.data: false
path.data: /data
network.bind_host: 10.100.16.9
network.publish_host: 10.100.16.9
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"
Master node elasticsearch.yml file
cluster.name: elkstats
node.name: Melasticsearch
node.master: true
node.data: true
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["Melasticsearch","datanode1","clientnode"]
path.data: /data
network.bind_host: 10.100.16.13
network.publish_host: 10.100.16.13
http.cors.enabled: true
http.cors.allow-origin: "*"
http.port: 9200
Data node elasticsearch.yml file
cluster.name: elkstats
node.name: datanode1
node.master: false
node.data: true
path.data: /data
network.bind_host: 10.100.16.11
network.publish_host: 10.100.16.11
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"
My cluster was working fine yesterday, but today after I installed Marvel, kibana shows:
there are 40000 lines in the /var/log/elasticsearch/elkstats.log, here I copied some:
[2016-05-12 16:48:28,076][WARN ][rest.suppressed ] /.marvel-es-1-/_field_stats Params: {level=indices, ignore_unavailable=true, index=.marvel-es-1-}
ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];]
at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:154)
at org.elasticsearch.action.fieldstats.TransportFieldStatsTransportAction.checkGlobalBlock(TransportFieldStatsTransportAction.java:182)
at org.elasticsearch.action.fieldstats.TransportFieldStatsTransportAction.checkGlobalBlock(TransportFieldStatsTransportAction.java:54)
at org.elasticsearch.action.support.broadcast.TransportBroadcastAction$AsyncBroadcastAction.(TransportBroadcastAction.java:118)
at org.elasticsearch.action.support.broadcast.TransportBroadcastAction.doExecute(TransportBroadcastAction.java:69)
...
