Firewall and other questions

Elastic Stack Elasticsearch
1

Hi

We're just testing elasticsearch and it is amazing.

We've set it up so that we have all ports closed except for port 80
and port 9300, and are reverse proxying with basic http authentication
to port 9200 on each host. (We're on EC2 running the zero conf cloud
magic plugin.) So far so good. Everything just works. It's a marvel.

However we're concerned and a little bit confused about the role that
9300 plays. Having 9300 open seems to be necessary to let the
different servers find each other in the first place, but if we close
it after they have found each other, the clustering carries on working
just fine. Is this correct?

The reason we're worried is we want to understand if it is OK to leave
9300 open at all or if we need to lock down firewall rules by IP or
similar. We're reluctant to do that as it is quite a bit of admin
overhead and fiddliness. I know that using something like Rightscale
it's possible to add a security group to itself - a sort of DIY VPC -
but as far as I'm aware that functionality doesn't exist on vanilla
AWS. We aren't sure if 9300 traffic is encrypted or signed or what.

I've had a look for more information on security but haven't had much
luck, although questions similar to this have been asked before - I
would have thought there are a lot of people who would benefit from
understanding the (lack of) security model so they can work out how to
set up a secure cluster in the wild.

Interested to know your thoughts,

David

2

The 9300 port is the port used for nodes to communicate with one another, not just for discovery. If you send a request to one node, and that request needs to happen on another node, then it will go through 9300. If you search a search request, and that search request needs to be executed on several nodes, it will go through 9300. The HTTP endpoint is just a gateway to the node, and from there on, it all goes through (the much more optimized) 9300 transport. So, it has to remain open.

The transport is not encrypted, though, it should not be difficult to add SSL to it. Though, I suggest against it. If you can close the ES cluster "within" itself, then its much better and faster then using SSL.
On Thursday, April 14, 2011 at 6:07 PM, David B. wrote:
Hi

We're just testing elasticsearch and it is amazing.

We've set it up so that we have all ports closed except for port 80
and port 9300, and are reverse proxying with basic http authentication
to port 9200 on each host. (We're on EC2 running the zero conf cloud
magic plugin.) So far so good. Everything just works. It's a marvel.

However we're concerned and a little bit confused about the role that
9300 plays. Having 9300 open seems to be necessary to let the
different servers find each other in the first place, but if we close
it after they have found each other, the clustering carries on working
just fine. Is this correct?

The reason we're worried is we want to understand if it is OK to leave
9300 open at all or if we need to lock down firewall rules by IP or
similar. We're reluctant to do that as it is quite a bit of admin
overhead and fiddliness. I know that using something like Rightscale
it's possible to add a security group to itself - a sort of DIY VPC -
but as far as I'm aware that functionality doesn't exist on vanilla
AWS. We aren't sure if 9300 traffic is encrypted or signed or what.

I've had a look for more information on security but haven't had much
luck, although questions similar to this have been asked before - I
would have thought there are a lot of people who would benefit from
understanding the (lack of) security model so they can work out how to
set up a secure cluster in the wild.

Interested to know your thoughts,

David

1 Like
3

I guess in that case that the connection must be being kept alive
somehow and not blocked by the firewall rules. Interesting.

As AWS newbie I've just worked out how to turn a group inwards on
itself as you suggest so sharing it here for someone else's
edification: in the security group config just choose the port that
you want to configure to talk only to machines in the same group
(9300) and enter the amazon-generated id of the security group itself
as the Source. Seems incredibly obvious now but took me a while to
figure out.

Thanks for the help.

On Apr 14, 5:17 pm, Shay Banon shay.ba...@elasticsearch.com wrote:

The 9300 port is the port used for nodes to communicate with one another, not just for discovery. If you send a request to one node, and that request needs to happen on another node, then it will go through 9300. If you search a search request, and that search request needs to be executed on several nodes, it will go through 9300. The HTTP endpoint is just a gateway to the node, and from there on, it all goes through (the much more optimized) 9300 transport. So, it has to remain open.

The transport is not encrypted, though, it should not be difficult to add SSL to it. Though, I suggest against it. If you can close the ES cluster "within" itself, then its much better and faster then using SSL.On Thursday, April 14, 2011 at 6:07 PM, David B. wrote:

Hi

We're just testing elasticsearch and it is amazing.

We've set it up so that we have all ports closed except for port 80
and port 9300, and are reverse proxying with basic http authentication
to port 9200 on each host. (We're on EC2 running the zero conf cloud
magic plugin.) So far so good. Everything just works. It's a marvel.

However we're concerned and a little bit confused about the role that
9300 plays. Having 9300 open seems to be necessary to let the
different servers find each other in the first place, but if we close
it after they have found each other, the clustering carries on working
just fine. Is this correct?

The reason we're worried is we want to understand if it is OK to leave
9300 open at all or if we need to lock down firewall rules by IP or
similar. We're reluctant to do that as it is quite a bit of admin
overhead and fiddliness. I know that using something like Rightscale
it's possible to add a security group to itself - a sort of DIY VPC -
but as far as I'm aware that functionality doesn't exist on vanilla
AWS. We aren't sure if 9300 traffic is encrypted or signed or what.

I've had a look for more information on security but haven't had much
luck, although questions similar to this have been asked before - I
would have thought there are a lot of people who would benefit from
understanding the (lack of) security model so they can work out how to
set up a secure cluster in the wild.

Interested to know your thoughts,

David

1 Like