# How to create elastic search cluster on AWS-EC2 using ES-6.5.2 **URL:** \ **Category:** Elasticsearch\ **Created:** [December 13, 2018, 5:43am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670 "2018-12-13T05:43:34Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 5:43am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/1 "2018-12-13T05:43:34Z")
Hi am creating cluster with 3 nodes making 2 nodes as master and data, 1 is data node 1. discovery.zen.ping.unicast.hosts: ["master\_node1:9300", "master\_node2:9300", "datanode:9300"] i have added like this in three nodes 2. node.name: node-1 node.master: true node.data: true node.ingest: true discovery.zen.hosts\_provider: ec2 network.host: 0.0.0.0 these are the configurations but i am able to see only one node i.e. master result: { "cluster\_name" : "Test\_ES", "status" : "green", "timed\_out" : false, "number\_of\_nodes" : 1, "number\_of\_data\_nodes" : 1, "active\_primary\_shards" : 0, "active\_shards" : 0, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 0, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 0, "number\_of\_in\_flight\_fetch" : 0, "task\_max\_waiting\_in\_queue\_millis" : 0, "active\_shards\_percent\_as\_number" : 100.0 } could you please help. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 13, 2018, 5:58am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/2 "2018-12-13T05:58:46Z")
> [@Madhu\_Cherukuri](#): > > making 2 nodes as master That's a bad idea. You should have 3 master nodes. If you define manually the unicast settings then remove this: ``` discovery.zen.hosts_provider: ec2 ``` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 13, 2018, 6:13am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/3 "2018-12-13T06:13:24Z")
If you have 2 or 3 master eligible nodes in the cluster, you must also set `discovery.zen.minimum_master_nodes` to `2` as per [these guidelines](https://www.elastic.co/guide/en/elasticsearch/https://www.elastic.co/guide/en/elasticsearch/reference/6.5/modules-node.html#split-brainreference/6.5/modules-node.html#split-brain). ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 6:25am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/4 "2018-12-13T06:25:35Z")
Hey David Pilato and Christian\_Dahlqvist Thanks for reply 1.I just installed the below plugin and i wrote this line "discovery.zen.hosts\_provider: ec2" cd /usr/share/elasticsearch/bin sudo ./elasticsearch-plugin install discovery-ec2 1. if i add "`discovery.zen.minimum_master_nodes` to `2`" the cluster will say there is no master node ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 6:27am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/5 "2018-12-13T06:27:17Z")
1. if i add " `discovery.zen.minimum_master_nodes` to `2` " the cluster will say there is no master node Because the connection between nodes has not established. I opened ports 9300, 9200 too ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 6:45am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/6 "2018-12-13T06:45:24Z")
## node-1 node.name: node-1 node.master: true node.data: true node.ingest: true #discovery.zen.hosts\_provider: ec2 network.host: 0.0.0.0 discovery.zen.ping.unicast.hosts: ["node-1", "node-2", "node-3"] #discovery.zen.minimum\_master\_nodes: 2 ## ==================================================================== node-2 node.name: node-2 node.master: true node.data: true node.ingest: true #discovery.zen.hosts\_provider: ec2 network.host: 0.0.0.0 discovery.zen.ping.unicast.hosts: ["node-1", "node-2", "node-3"] #discovery.zen.minimum\_master\_nodes: 2 ## ================================================================== node-3 node.name: node-3 node.master: false node.data: true node.ingest: true #discovery.zen.hosts\_provider: ec2 network.host: 0.0.0.0 discovery.zen.ping.unicast.hosts: ["node-1", "node-2", "node-3"] #discovery.zen.minimum\_master\_nodes: 2 if i curl from node-1 OUTPUT: curl -XGET [http://localhost:9200/\_cluster/health?pretty](http://localhost:9200/_cluster/health?pretty) { "cluster\_name" : "Test\_ES", "status" : "green", "timed\_out" : false, "number\_of\_nodes" : 1, "number\_of\_data\_nodes" : 1, "active\_primary\_shards" : 0, "active\_shards" : 0, "relocating\_shards" : 0, "initializing\_shards" : 0, "unassigned\_shards" : 0, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 0, "number\_of\_in\_flight\_fetch" : 0, "task\_max\_waiting\_in\_queue\_millis" : 0, "active\_shards\_percent\_as\_number" : 100.0 } Please help me to add nodes to this cluster. Thanks. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 13, 2018, 6:49am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/7 "2018-12-13T06:49:52Z")
As David pointed out, having 2 master-eligible nodes is bad and you should make all 3 master-eligible. > [@Madhu\_Cherukuri](#): > > #discovery.zen.minimum\_master\_nodes: 2 You should not have this commented out. > [@Madhu\_Cherukuri](#): > > discovery.zen.ping.unicast.hosts: ["node-1", "node-2", "node-3"] Does this resolve to the IP addresses of the nodes in the cluster? Are you able to telnet to port 9300 between the hosts? ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 7:11am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/8 "2018-12-13T07:11:25Z")
I made all three as master nodes and "discovery.zen.minimum\_master\_nodes: 2" added that line telnet IP 9300 Trying IP... Connected to IP. Escape character is '^]'. now OUTPUT: { "error" : { "root\_cause" : [ { "type" : "master\_not\_discovered\_exception", "reason" : null } ], "type" : "master\_not\_discovered\_exception", "reason" : null }, "status" : 503 } ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 7:21am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/9 "2018-12-13T07:21:17Z")
this is the node-1 log [2018-12-13T00:39:17,007][INFO][o.e.d.z.ZenDiscovery] [node-1] failed to send join request to master [{node-2}{KaO-RmXBQYKDukApaHOW8g}{L9ODM6TdTUmp5JNlyM7mKg}{IP}{IP:9300}{ml.machine\_memory=4135096320, ml.max\_open\_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-2][IP:9300][internal:discovery/zen/join]]; nested: IllegalArgumentException[can't add node {node-1}{KaO-RmXBQYKDukApaHOW8g}{RB-Oa2GNTyON1wy5UC0aPQ}{IP}{IP:9300}{ml.machine\_memory=4135108608, ml.max\_open\_jobs=20, xpack.installed=true, ml.enabled=true}, found existing node {node-2}{KaO-RmXBQYKDukApaHOW8g}{L9ODM6TdTUmp5JNlyM7mKg}{IP}{IP:9300}{ml.machine\_memory=4135096320, xpack.installed=true, ml.max\_open\_jobs=20, ml.enabled=true} with the same id but is a different node instance]; ] ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 13, 2018, 7:21am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/10 "2018-12-13T07:21:48Z")
Are you specifying IP or hostname in `discovery.zen.ping.unicast.hosts`? ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 7:22am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/11 "2018-12-13T07:22:26Z")
now i changed to IP ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 7:44am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/12 "2018-12-13T07:44:34Z")
Same output { "error" : { "root\_cause" : [ { "type" : "master\_not\_discovered\_exception", "reason" : null } ], "type" : "master\_not\_discovered\_exception", "reason" : null }, "status" : 503 } No luck as of now LOGS: [2018-12-13T01:16:29,714][INFO][o.e.d.z.ZenDiscovery] [node-1] failed to send join request to master [{node-2}{KaO-RmXBQYKDukApaHOW8g}{L9ODM6TdTUmp5JNlyM7mKg}{IP}{IP:9300}{ml.machine\_memory=4135096320, ml.max\_open\_jobs=20, xpack.installed=true, ml.enabled=true}], reason [RemoteTransportException[[node-2][IP:9300][internal:discovery/zen/join]]; nested: IllegalArgumentException[can't add node {node-1}{KaO-RmXBQYKDukApaHOW8g}{RB-Oa2GNTyON1wy5UC0aPQ}{IP}{IP:9300}{ml.machine\_memory=4135108608, ml.max\_open\_jobs=20, xpack.installed=true, ml.enabled=true}, found existing node {node-2}{KaO-RmXBQYKDukApaHOW8g}{L9ODM6TdTUmp5JNlyM7mKg}{IP}{IP:9300}{ml.machine\_memory=4135096320, xpack.installed=true, ml.max\_open\_jobs=20, ml.enabled=true} with the same id but is a different node instance]; ] Any help? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 13, 2018, 7:48am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/13 "2018-12-13T07:48:08Z")
> [@Madhu\_Cherukuri](#): > > found existing node {node-2}{KaO-RmXBQYKDukApaHOW8g}{L9ODM6TdTUmp5JNlyM7mKg}{IP}{IP:9300}{ml.machine\_memory=4135096320, xpack.installed=true, ml.max\_open\_jobs=20, ml.enabled=true} with the same id but is a different node instance How did you install Elasticsearch on the hosts? Did you by any chance just copy the disks or directories instead of installing from scratch? ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 7:51am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/14 "2018-12-13T07:51:32Z")
I installed from scratch on node1, after I took image and launched two more instances, After I cahnged config files on each instance. ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 7:53am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/15 "2018-12-13T07:53:48Z")
1. So you mean to say ID's are same, do I need to install from scratch on each server? 2. Also de we need any plugins for Aws specifically? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 13, 2018, 8:00am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/16 "2018-12-13T08:00:07Z")
I believe the ID is generated when the node first starts up, so I assume you started Elasticsearch on the node before you took the image? This would in that case explain the issue. restarting while clearing the data directory and should force a new ID to be generated. There may be easier ways to correct this, but I have not encountered this before. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [December 13, 2018, 10:19am UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/17 "2018-12-13T10:19:24Z")
> Also de we need any plugins for Aws specifically? If you want to use automatic discovery and configure `network.host` based on `_ec2_` IP addresses, yes. Otherwise just remove the `discovery-ec2` plugin. ---
**Author:** ![Madhu\_Cherukuri](https://avatars.discourse-cdn.com/v4/letter/m/3ab097/32.png) [@Madhu\_Cherukuri](https://discuss.elastic.co/u/Madhu_Cherukuri)\ **Post date:** [December 13, 2018, 12:34pm UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/18 "2018-12-13T12:34:47Z")
Thanks Christian and David Error was same Id, on all nodes, because of installation. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 10, 2019, 12:34pm UTC](https://discuss.elastic.co/t/how-to-create-elastic-search-cluster-on-aws-ec2-using-es-6-5-2/160670/19 "2019-01-10T12:34:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.