# Connect to EC2 cluster from other security groups **URL:** \ **Category:** Elasticsearch\ **Created:** [February 9, 2012, 6:29am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653 "2012-02-09T06:29:34Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![vreal](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vreal](https://discuss.elastic.co/u/vreal)\ **Post date:** [February 9, 2012, 6:29am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/1 "2012-02-09T06:29:34Z")
I've created a cluster with 2 EC2 instances on amazon and set the security group as search-cluster and they are works fine. Then I start another instance which is in security group web-server. Both security groups have port 9200 and 9300 opened. In this web server i have a web app which contains such code: Builder settings = ImmutableSettings.settingsBuilder(); settings.put("discovery.type", "ec2"); settings.put("cloud.aws.access\_key", "my access key"); settings.put("cloud.aws.secret\_key", "my security key"); settings.put("discovery.ec2.groups", "search-cluster"); Node node = nodeBuilder().settings(settings).clusterName("my cluster name").client(true).node(); client = node.client(); Then I got such error when I index a record into the elasticsearch: org.elasticsearch.discovery.MasterNotDiscoveredException: org.elasticsearch.action.support.master.TransportMasterNodeOperationAction$3.onTimeout(TransportMasterNodeOperationAction.java:162) org.elasticsearch.cluster.service.InternalClusterService$NotifyTimeout.run(InternalClusterService.java:332) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) java.lang.Thread.run(Thread.java:636) Can anyone tell me what can I do to deal with the problem? Thanks ---
**Author:** ![vreal](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vreal](https://discuss.elastic.co/u/vreal)\ **Post date:** [February 9, 2012, 7:01am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/2 "2012-02-09T07:01:27Z")
The elasticsearch is version 0.18.7 and java 1.6 ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 12, 2012, 4:43pm UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/3 "2012-02-12T16:43:47Z")
Set the org.elasticsearch.discovery to TRACE and you will see which nodes are used to join the cluster. You might need to provide both groups as well in the configuration. On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > The elasticsearch is version 0.18.7 and java 1.6 > > -- > View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com) ([http://Nabble.com](http://Nabble.com)). ---
**Author:** ![vreal](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vreal](https://discuss.elastic.co/u/vreal)\ **Post date:** [February 13, 2012, 7:05am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/4 "2012-02-13T07:05:48Z")
Did you mean set the setting in java code? I've tried but didn't see the TRACE log. On the other hand I've tried to On 2012/02/13, at 0:43 , Shay Banon wrote: > Set the org.elasticsearch.discovery to TRACE and you will see which nodes are used to join the cluster. You might need to provide both groups as well in the configuration. > On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > > > The elasticsearch is version 0.18.7 and java 1.6 > > > > -- > > View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![vreal](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vreal](https://discuss.elastic.co/u/vreal)\ **Post date:** [February 14, 2012, 6:09am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/5 "2012-02-14T06:09:54Z")
I provide more info. 1. I'm running elasticsearch client in amazon's mapreduce process and we were using cascading([www.cascading.org](http://www.cascading.org)) to help us programming mapreduce programs. 2. The mapreduct process is running in the same region but different groups: ElasticMapReduce-slave, ElasticMapReduce-master. I've opened port 9200 and 9300 on both of the group 3. I've changed the log setting and tested locally and trace log has been printed out but it didn't print any trace log after I've upload the package on amazon mapreduce environment. The attachment is the log we get from the result. Can anyone give any advice? Thanks, Ye Zhou On 2012/02/13, at 15:5 , Ye Zhou wrote: > Did you mean set the setting in java code? I've tried but didn't see the TRACE log. > > On the other hand I've tried to > > On 2012/02/13, at 0:43 , Shay Banon wrote: > > > Set the org.elasticsearch.discovery to TRACE and you will see which nodes are used to join the cluster. You might need to provide both groups as well in the configuration. > > On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > > > > > The elasticsearch is version 0.18.7 and java 1.6 > > > > > > -- > > > View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [February 14, 2012, 6:16pm UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/6 "2012-02-14T18:16:25Z")
Difficult to tell what is happening since you are running from inside Hadoop. How are you creating the ES client? Are you sure it is created fully on each Hadoop node? -- Ivan On Mon, Feb 13, 2012 at 10:09 PM, Ye Zhou [zhouy.vreal@gmail.com](mailto:zhouy.vreal@gmail.com) wrote: > I provide more info. > > 1. I'm running elasticsearch client in amazon's mapreduce process and we > were using cascading([www.cascading.org](http://www.cascading.org)) to help us programming mapreduce > programs. > 2. The mapreduct process is running in the same region but different > groups: ElasticMapReduce-slave, ElasticMapReduce-master. I've opened port > 9200 and 9300 on both of the group > 3. I've changed the log setting and tested locally and trace log has been > printed out but it didn't print any trace log after I've upload the package > on amazon mapreduce environment. > > The attachment is the log we get from the result. Can anyone give any > advice? > > Thanks, > > Ye Zhou > > On 2012/02/13, at 15:5 , Ye Zhou wrote: > > Did you mean set the setting in java code? I've tried but didn't see the > TRACE log. > > On the other hand I've tried to > > On 2012/02/13, at 0:43 , Shay Banon wrote: > > Set the org.elasticsearch.discovery to TRACE and you will see which nodes > are used to join the cluster. You might need to provide both groups as well > in the configuration. > > On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > > The elasticsearch is version 0.18.7 and java 1.6 > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![vreal](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vreal](https://discuss.elastic.co/u/vreal)\ **Post date:** [February 15, 2012, 12:58am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/7 "2012-02-15T00:58:50Z")
Hi, Here's my code: > > Builder settings = ImmutableSettings.settingsBuilder(); > > settings.put("discovery.type", "ec2"); > > settings.put("cloud.aws.access\_key", "my access key"); > > settings.put("cloud.aws.secret\_key", "my security key"); > > settings.put("discovery.ec2.groups", "search-cluster"); > > Node node = nodeBuilder().settings(settings).clusterName("my cluster name").client(true).node(); > > client = node.client(); Regards, Ye Zhou On 2012/02/15, at 2:16 , Ivan Brusic wrote: > Difficult to tell what is happening since you are running from inside > Hadoop. How are you creating the ES client? Are you sure it is created > fully on each Hadoop node? > > -- > Ivan > > On Mon, Feb 13, 2012 at 10:09 PM, Ye Zhou [zhouy.vreal@gmail.com](mailto:zhouy.vreal@gmail.com) wrote: > > > I provide more info. > > > > 1. I'm running elasticsearch client in amazon's mapreduce process and we > > were using cascading([www.cascading.org](http://www.cascading.org)) to help us programming mapreduce > > programs. > > 2. The mapreduct process is running in the same region but different > > groups: ElasticMapReduce-slave, ElasticMapReduce-master. I've opened port > > 9200 and 9300 on both of the group > > 3. I've changed the log setting and tested locally and trace log has been > > printed out but it didn't print any trace log after I've upload the package > > on amazon mapreduce environment. > > > > The attachment is the log we get from the result. Can anyone give any > > advice? > > > > Thanks, > > > > Ye Zhou > > > > On 2012/02/13, at 15:5 , Ye Zhou wrote: > > > > Did you mean set the setting in java code? I've tried but didn't see the > > TRACE log. > > > > On the other hand I've tried to > > > > On 2012/02/13, at 0:43 , Shay Banon wrote: > > > > Set the org.elasticsearch.discovery to TRACE and you will see which nodes > > are used to join the cluster. You might need to provide both groups as well > > in the configuration. > > > > On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > > > > The elasticsearch is version 0.18.7 and java 1.6 > > > > -- > > View this message in context: > > [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![vreal](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vreal](https://discuss.elastic.co/u/vreal)\ **Post date:** [February 16, 2012, 8:45am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/8 "2012-02-16T08:45:04Z")
Hi, It sounds solved. At beginning we just opened ports 9200 and 9300 on security groups but now we opened ports from 9200 to 9399. But from the document it seems it just require port 9200 and 9300. Can anyone explain? Regards, Ye Zhou On 2012/02/15, at 8:58 , Ye Zhou wrote: > Hi, > > Here's my code: > > > > Builder settings = ImmutableSettings.settingsBuilder(); > > > settings.put("discovery.type", "ec2"); > > > settings.put("cloud.aws.access\_key", "my access key"); > > > settings.put("cloud.aws.secret\_key", "my security key"); > > > settings.put("discovery.ec2.groups", "search-cluster"); > > > Node node = nodeBuilder().settings(settings).clusterName("my cluster name").client(true).node(); > > > client = node.client(); > > Regards, > Ye Zhou > > On 2012/02/15, at 2:16 , Ivan Brusic wrote: > > > Difficult to tell what is happening since you are running from inside > > Hadoop. How are you creating the ES client? Are you sure it is created > > fully on each Hadoop node? > > > > -- > > Ivan > > > > On Mon, Feb 13, 2012 at 10:09 PM, Ye Zhou [zhouy.vreal@gmail.com](mailto:zhouy.vreal@gmail.com) wrote: > > > > > I provide more info. > > > > > > 1. I'm running elasticsearch client in amazon's mapreduce process and we > > > were using cascading([www.cascading.org](http://www.cascading.org)) to help us programming mapreduce > > > programs. > > > 2. The mapreduct process is running in the same region but different > > > groups: ElasticMapReduce-slave, ElasticMapReduce-master. I've opened port > > > 9200 and 9300 on both of the group > > > 3. I've changed the log setting and tested locally and trace log has been > > > printed out but it didn't print any trace log after I've upload the package > > > on amazon mapreduce environment. > > > > > > The attachment is the log we get from the result. Can anyone give any > > > advice? > > > > > > Thanks, > > > > > > Ye Zhou > > > > > > On 2012/02/13, at 15:5 , Ye Zhou wrote: > > > > > > Did you mean set the setting in java code? I've tried but didn't see the > > > TRACE log. > > > > > > On the other hand I've tried to > > > > > > On 2012/02/13, at 0:43 , Shay Banon wrote: > > > > > > Set the org.elasticsearch.discovery to TRACE and you will see which nodes > > > are used to join the cluster. You might need to provide both groups as well > > > in the configuration. > > > > > > On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > > > > > > The elasticsearch is version 0.18.7 and java 1.6 > > > > > > -- > > > View this message in context: > > > [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 16, 2012, 7:49pm UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/9 "2012-02-16T19:49:18Z")
Only 9200 and 9300 are needed, unless you start several instances on the same machine. Check the log, it logs the ports that are opened. On Thursday, February 16, 2012 at 10:45 AM, Ye Zhou wrote: > Hi, > > It sounds solved. At beginning we just opened ports 9200 and 9300 on security groups but now we opened ports from 9200 to 9399. But from the document it seems it just require port 9200 and 9300. Can anyone explain? > > Regards, > Ye Zhou > > On 2012/02/15, at 8:58 , Ye Zhou wrote: > > > Hi, > > > > Here's my code: > > > > > > Builder settings = ImmutableSettings.settingsBuilder(); > > > > settings.put("discovery.type", "ec2"); > > > > settings.put("cloud.aws.access\_key", "my access key"); > > > > settings.put("cloud.aws.secret\_key", "my security key"); > > > > settings.put("discovery.ec2.groups", "search-cluster"); > > > > Node node = nodeBuilder().settings(settings).clusterName("my cluster name").client(true).node(); > > > > client = node.client(); > > > > Regards, > > Ye Zhou > > > > On 2012/02/15, at 2:16 , Ivan Brusic wrote: > > > > > Difficult to tell what is happening since you are running from inside > > > Hadoop. How are you creating the ES client? Are you sure it is created > > > fully on each Hadoop node? > > > > > > -- > > > Ivan > > > > > > On Mon, Feb 13, 2012 at 10:09 PM, Ye Zhou \<[zhouy.vreal@gmail.com](mailto:zhouy.vreal@gmail.com) ([mailto:zhouy.vreal@gmail.com](mailto:zhouy.vreal@gmail.com))\> wrote: > > > > > > > I provide more info. > > > > > > > > 1. I'm running elasticsearch client in amazon's mapreduce process and we > > > > were using cascading([www.cascading.org](http://www.cascading.org) ([http://www.cascading.org/](http://www.cascading.org/))) to help us programming mapreduce > > > > programs. > > > > 2. The mapreduct process is running in the same region but different > > > > groups: ElasticMapReduce-slave, ElasticMapReduce-master. I've opened port > > > > 9200 and 9300 on both of the group > > > > 3. I've changed the log setting and tested locally and trace log has been > > > > printed out but it didn't print any trace log after I've upload the package > > > > on amazon mapreduce environment. > > > > > > > > The attachment is the log we get from the result. Can anyone give any > > > > advice? > > > > > > > > Thanks, > > > > > > > > Ye Zhou > > > > > > > > On 2012/02/13, at 15:5 , Ye Zhou wrote: > > > > > > > > Did you mean set the setting in java code? I've tried but didn't see the > > > > TRACE log. > > > > > > > > On the other hand I've tried to > > > > > > > > On 2012/02/13, at 0:43 , Shay Banon wrote: > > > > > > > > Set the org.elasticsearch.discovery to TRACE and you will see which nodes > > > > are used to join the cluster. You might need to provide both groups as well > > > > in the configuration. > > > > > > > > On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > > > > > > > > The elasticsearch is version 0.18.7 and java 1.6 > > > > > > > > -- > > > > View this message in context: > > > > [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > > > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com) ([http://Nabble.com/](http://Nabble.com/)). ---
**Author:** ![vreal](https://avatars.discourse-cdn.com/v4/letter/v/5f9b8f/32.png) [@vreal](https://discuss.elastic.co/u/vreal)\ **Post date:** [February 17, 2012, 5:27am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/10 "2012-02-17T05:27:01Z")
Thanks, I found the logs has the info that's why I've tried to open more ports. When we start the Hadoop Mapreduce process it seems started more than one instances on the same machine. I think that's the reason. Regads, Bruce Zhou On 2012/02/17, at 3:49 , Shay Banon wrote: > Only 9200 and 9300 are needed, unless you start several instances on the same machine. Check the log, it logs the ports that are opened. > On Thursday, February 16, 2012 at 10:45 AM, Ye Zhou wrote: > > > Hi, > > > > It sounds solved. At beginning we just opened ports 9200 and 9300 on security groups but now we opened ports from 9200 to 9399. But from the document it seems it just require port 9200 and 9300. Can anyone explain? > > > > Regards, > > Ye Zhou > > > > On 2012/02/15, at 8:58 , Ye Zhou wrote: > > > > > Hi, > > > > > > Here's my code: > > > > > > > > Builder settings = ImmutableSettings.settingsBuilder(); > > > > > settings.put("discovery.type", "ec2"); > > > > > settings.put("cloud.aws.access\_key", "my access key"); > > > > > settings.put("cloud.aws.secret\_key", "my security key"); > > > > > settings.put("discovery.ec2.groups", "search-cluster"); > > > > > Node node = nodeBuilder().settings(settings).clusterName("my cluster name").client(true).node(); > > > > > client = node.client(); > > > > > > Regards, > > > Ye Zhou > > > > > > On 2012/02/15, at 2:16 , Ivan Brusic wrote: > > > > > > > Difficult to tell what is happening since you are running from inside > > > > Hadoop. How are you creating the ES client? Are you sure it is created > > > > fully on each Hadoop node? > > > > > > > > -- > > > > Ivan > > > > > > > > On Mon, Feb 13, 2012 at 10:09 PM, Ye Zhou [zhouy.vreal@gmail.com](mailto:zhouy.vreal@gmail.com) wrote: > > > > > > > > > I provide more info. > > > > > > > > > > 1. I'm running elasticsearch client in amazon's mapreduce process and we > > > > > were using cascading([www.cascading.org](http://www.cascading.org)) to help us programming mapreduce > > > > > programs. > > > > > 2. The mapreduct process is running in the same region but different > > > > > groups: ElasticMapReduce-slave, ElasticMapReduce-master. I've opened port > > > > > 9200 and 9300 on both of the group > > > > > 3. I've changed the log setting and tested locally and trace log has been > > > > > printed out but it didn't print any trace log after I've upload the package > > > > > on amazon mapreduce environment. > > > > > > > > > > The attachment is the log we get from the result. Can anyone give any > > > > > advice? > > > > > > > > > > Thanks, > > > > > > > > > > Ye Zhou > > > > > > > > > > On 2012/02/13, at 15:5 , Ye Zhou wrote: > > > > > > > > > > Did you mean set the setting in java code? I've tried but didn't see the > > > > > TRACE log. > > > > > > > > > > On the other hand I've tried to > > > > > > > > > > On 2012/02/13, at 0:43 , Shay Banon wrote: > > > > > > > > > > Set the org.elasticsearch.discovery to TRACE and you will see which nodes > > > > > are used to join the cluster. You might need to provide both groups as well > > > > > in the configuration. > > > > > > > > > > On Thursday, February 9, 2012 at 9:01 AM, vreal wrote: > > > > > > > > > > The elasticsearch is version 0.18.7 and java 1.6 > > > > > > > > > > -- > > > > > View this message in context: > > > > > [http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html](http://elasticsearch-users.115913.n3.nabble.com/Connect-to-EC2-cluster-from-other-security-groups-tp3728579p3728629.html) > > > > > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:38am UTC](https://discuss.elastic.co/t/connect-to-ec2-cluster-from-other-security-groups/6653/11 "2017-07-06T03:38:57Z")