Deploying a 3 nodes cluster with ansible

Elastic Stack Elasticsearch
1

Hi !

I would like to deploy a 3 nodes cluster with ansible. (I'm new in the ansible world)
I would like to know if the playbook I made will answer all my requests.

My requests are:

  • deploy elasticsearch and logstash to 3 servers.
  • whenever a server break, I just need to launch the playbook so a new elasticsearch and logstash instance are installed in the new server and are added to the current cluster. (i don't backup the servers, so I need to reinstall when one break)

The roles I use are the official elacticsearch roles and for logstash: https://github.com/CyVerse-Ansible/ansible-logstash

Here is my playbook: https://gist.github.com/newclem/494d99c3815b1d32f15a6f038f0c92f1

Thank you !

2

You should test whether it works. I can offer a few comments:

discovery.zen.ping.unicast.hosts: "liprd072:9300,liprd073,9300",

The last comma should of course be a colon.

You shouldn't have three nearly identical plays in the playbook. Use a single play that acts on all hosts. If they need different variables then use group_vars and/or host_vars to use different values for each host.

I'm pretty sure that discovery.zen.ping.unicast.hosts can always list all hostnames, i.e. you don't have to be picky about not listing a node's own hostname.

You're listing discovery.zen.ping.multicast.enabled twice for each es_config role.

3

Thank you for your reply.

I've updated my play: https://gist.github.com/newclem/494d99c3815b1d32f15a6f038f0c92f1

And in the inventory I put a host var. hosts_unicast=liprd071:9300,liprd71:9300 (each var are different for each host)

Am I doing it right ?

4

Yeah, you're on the right track.

Note that the groups dictionary contains a mapping of all inventory groups and their members, so instead of defining a separate variable that lists the hostnames and referencing it when you set the unicast host list you can say groups['nodes'].

But this is more about Ansible than Elasticsearch so I'll stop here.

5

Okay thank you for your reply.

I launched the playbook and everything have been installed but I have now 2 problems:

  1. There is no elasticsearch service. Elastic search is launched with and not as a service
  2. My nodes are alone, the unicast discover doesn't work. I have 3 nodes but there not in the same cluster. How do I make it works ?

Thank you !

6
  1. I don't understand the question. There appears to be a word missing.
  2. What's in the ES logs? Is ES listening to the right interfaces? As of 2.0 it's only listening on localhost by default.
7

  1. Oups sorry, the word was Java.

  2. I put network.host: eth0

    [2016-10-20 10:34:10,837][INFO ][node ] [LIPRD071-node1] version[2.4.1], pid[25300], build[c67dc32/2016-09-27T18:57:55Z]
    [2016-10-20 10:34:10,838][INFO ][node ] [LIPRD071-node1] initializing ...
    [2016-10-20 10:34:11,869][INFO ][plugins ] [LIPRD071-node1] modules [lang-groovy, reindex, lang-expression], plugins [], sites []
    [2016-10-20 10:34:11,908][INFO ][env ] [LIPRD071-node1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [484.6gb], net total_space [506.8gb], spins? [unknown], types [rootfs]
    [2016-10-20 10:34:11,909][INFO ][env ] [LIPRD071-node1] heap size [2.9gb], compressed ordinary object pointers [true]
    [2016-10-20 10:34:15,052][INFO ][node ] [LIPRD071-node1] initialized
    [2016-10-20 10:34:15,052][INFO ][node ] [LIPRD071-node1] starting ...
    [2016-10-20 10:34:15,304][INFO ][transport ] [LIPRD071-node1] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
    [2016-10-20 10:34:15,311][INFO ][discovery ] [LIPRD071-node1] ES-cluster/olHth8g4RJG6zvhFMdpCpg
    [2016-10-20 10:34:18,354][INFO ][cluster.service ] [LIPRD071-node1] new_master {LIPRD071-node1}{olHth8g4RJG6zvhFMdpCpg}{127.0.0.1}{127.0.0.1:9300}{master=true}, reason: zen-disco-join(elected_as_master, [0] joins received)
    [2016-10-20 10:34:18,379][INFO ][http ] [LIPRD071-node1] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
    [2016-10-20 10:34:18,380][INFO ][node ] [LIPRD071-node1] started
    [2016-10-20 10:34:18,427][INFO ][gateway ] [LIPRD071-node1] recovered [0] indices into cluster_state

If read well, I should put a publish_adress wich is the ip adress of the server ?

8
  1. Okay. It sounds weird that the playbook doesn't start ES via service elasticsearch start but I don't have time to dig into this.
  2. Your logs indicate that you're only binding to localhost.
9
  1. In fact the service elasticsearch doesn't exists but a script called node1-elasticsearch exists in /etc/default. (or node2-elasticsearch, etc.)
  2. Okay I'll set this up and I'll come back. Thank you

EDIT: I don't know how to bind to another adress. I put network.host: site and it still bind to localhost. Do i have do specify the IP adress ?

EDIT2: The log are not up to date, I don't know why. And I can't apt-get remove elasticsearch... Something doesn't work well..

EDIT3: All my servers are down, I have to rebuild them.