I'm a noob at subnets / network masks and can't get my nodes to discover each other

Elastic Stack Elasticsearch
1

Can someone please tell me what I'm doing wrong? I'm using three Amazon EC2 servers and I just want them to discover each other and form a cluster. They are part of a private subnet.

How should I add this to their elasticsearch.yml configs?

Also, I'm a bit confused about the network.host value. Should all three nodes be left at their default of local? Or do I need to find one node and keep that as local then set the other nodes to have that node's ip address?

Thank you

2

Hey!

I recommend reading this section in the elastic docs!

By default, elastic instances will talk to each other via port 9200 and 9300, so if you have security groups around those EC2 servers, you will have to open those ports.

A simple test would be to start elastic on machine X and from machine Y do

$> telnet IP_OF_X 9300

and see if you get a connection.

For my personal cluster in AWS, I have the following setting for EC2 nodes which are in a private subnet as well:

network:
  host: "0.0.0.0"
  publish_host: ""_ec2:privateIpv4_"
...
...
discovery:
  host_type: "private_ip"
3

Okay, I'll try this. I also set up Nginx as a reverse proxy on the / location block with port 9200, so maybe that is causing an issue? I didn't do anything with 9300.

If I ssh into one of the EC2 servers I am able to ping another one of the servers' IP addresses.

What is the _ec2 notation? I'm not familiar with that.

4

Also, does this mean we need to use this discovery-ec2 plugin or can we configure it without the plugin?

5

I am not sure why you need a proxy for inter node communication :frowning:

ping usually uses the ICMP protocol which can be enabled separately to the TCP protocol on AWS security groups. You need to check if you can talk on port 9200/9300 between nodes.

Based on this comment and your post below, I think you need to look up the AWS EC2 discovery plugin and read a bit more about it :slight_smile:

6

I actually figured it out! It was an issue with my elasticsearch.yml config. I got through setting up everything else without many issues, but the networking aspects of configuring a cluster was the most confusing bits for me since teaching myself Elasticsearch stuff, probably since I don't have a computer science degree. I never have to deal with networking stuff before, so even though I figured it out, I'm still a bit foggy as to what is going on.

It was regarding network.host and the zen discovery field. I got it working by setting network.host to 0.0.0.0 and adding the URIs of the other nodes to the zen discovery array. The documentation on these fields doesn't, in my opinion, make it very clear to people who are unfamiliar with networking or Elasticsearch, but that could just be me.

What I'm still wondering (if you could help clarify) is:

  1. Why does network.host have to be set to 0.0.0.0 instead of the default value of local? I believe when I commented that like out that the other ports weren't accessible via http requests, which is why it wasn't forming a cluster (I think).

  2. When I add another node to the cluster, do I have to add it to the zen discovery array of all the others, and add all the others to its array?

  3. Now that I have a cluster up and running, which node do I use for communicating with (making queries, sending commands, etc)?

Thank you for your help!

7

Let no one tell you that you need that to do anything with elastic or even programming! You can invest your time and effort into learning programming and/or using elastic without that and still do excellently well.

Usually when you specify the host as local, it means any connection originating from the ip 127.0.0.1, which means that it will allow any connections as long as they are from the same machine/box! What we want is that it allows any connection on that port, regardless of where it came from! Which is where the 0.0.0.0 comes in. By telling elastic to accept connections on this interface, you are telling it to accept connections that may come from outside this machine itself, which is what you want in cluster.

Did you have a look at the aws plugin for elastic? You should not have to add any URI's anywhere and it should discover nodes on its own. Look up the plugin and also look up discovery via EC2 tags.

You can do that against any node and it will work. The beauty of elastic :slight_smile:

Next I suggest reading up on the different kinds of nodes and their purpose. :slight_smile:

Best of luck.

8

Thank you very much! That is very helpful. I will do that! =)

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.