Simple Elasticsearch cluster with 2 data nodes


(Alex Harvey) #1

Hi all

I am struggling a bit with the ES 2.2 documentation on clustering. Seems like there are many ways of building ES clusters and lots of blog posts out there from different stages of this project's history but a bit hard to find what's a recommended, simple configuration.

I simply want 2 nodes in an active/active configuration that will be easy to scale out horizontally. The ability to auto-scale isn't a requirement. It is to be part of an ELK stack.

Many thanks,


(David Pilato) #2

Just start 2 nodes. Et voila !

But you should add a 3rd one as a Tie breaker in case of network loss and set minimum master node.


(Alex Harvey) #3

Any chance you could give me a hint on how to configure the nodes before I start them? I've inherited a configuration that's using the Zen Discovery module and it's all proving to be anything but easy.


(David Pilato) #4

Basically you have to set network.host and the unicast list of nodes.

David.


(Alex Harvey) #5

I should have mentioned, this ES cluster is to be part of an ELK stack.


(Alex Harvey) #6

Thanks so much David.

I ended up with this:

---
cluster:
  name: es01
discovery:
  zen:
    ping:
      unicast:
        hosts:
             - data01.mydomain.com:9300
             - data02.mydomain.com:9300
http:
  enabled: true
index:
  gateway:
    type: none
  number_of_replicas: 1
  number_of_shards: 5
indices:
  fielddata:
    breaker:
      limit: 60%
    cache:
      size: 100%
network:
  host: 1.1.1.1
node:
  data: true
  master: true
  name: es01_data01
path:
  data: /srv/es
  logs: /var/log/elasticsearch/es01

(David Pilato) #7

Looks good. I'd simplify it though with:

cluster:
  name: es01
discovery:
  zen:
    ping:
      unicast:
        hosts:
             - data01.mydomain.com:9300
             - data02.mydomain.com:9300
network:
  host: 1.1.1.1
node:
  name: es01_data01
path:
  data: /srv/es
  logs: /var/log/elasticsearch/es01

(Alex Harvey) #8

Great, thanks again. I'll write yet another blog post and document all this when I'm finished!


(Alex Harvey) #9

Is it possible to specify the same configuration in Logstash? If the version of logstash is 1.4.5? I ended up making the 1.4.5 choice because Elastic.co's Puppet module only supports Logstash 1.4.


(system) #10