Setting ElasticSearch 5.5.0 Cluster With discovery-ec2 Plugin

Elastic Stack Elasticsearch
1

Hello,

I have Configured ElasticSearch Cluster (3 Nodes) with the discovery-ec2 plugin and my configuration below:

ES1 (Master Eligible)

cluster.name: my-cluster
path.data: /data/data
path.logs: /data/logs
network.host: _ec2_
node.data: true
node.master: true
cloud.node.auto_attributes: true
discovery.ec2.tag.es_cluster: prod-elasticsearch
discovery.zen.hosts_provider: ec2
cluster.routing.allocation.awareness.attributes: aws_availability_zone
cloud.aws.region: eu-west-1

ES2 (Master Eligible)

cluster.name: my-cluster
path.data: /data/data
path.logs: /data/logs
network.host: _ec2_
node.data: true
node.master: true
cloud.node.auto_attributes: true
discovery.ec2.tag.es_cluster: prod-elasticsearch
discovery.zen.hosts_provider: ec2
cluster.routing.allocation.awareness.attributes: aws_availability_zone
cloud.aws.region: eu-west-1

ES3

cluster.name: my-cluster
path.data: /data/data
path.logs: /data/logs
network.host: _ec2_
node.data: true
node.master: false
cloud.node.auto_attributes: true
discovery.zen.hosts_provider: ec2
discovery.ec2.tag.es_cluster: prod-elasticsearch
cluster.routing.allocation.awareness.attributes: aws_availability_zone
cloud.aws.region: eu-west-1

It is working fine with out creating EC2 Tags in AWS Console.

Should I Create the TAG in AWS Console?
Can I go live with the configuration?

2

Hello, Can anybody help on this?

3

I formated your initial post.

It is working fine with out creating EC2 Tags in AWS Console.

So you mean that discovery is working well even though you did not set any AWS tag for your instances?

If so, then there is something wrong. I need to look at the code then.
But can you confirm that first?

Can you run GET _cat/nodes?v?

4 
[root@ip-10-3-4-73 ec2-user]#  curl -XGET http://10.3.2.163:9200/_cat/nodes?v
ip         heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.3.0.217            4          59   0    0.00    0.00     0.00 mdi       -      jixD7ms
10.3.4.73             3          60   0    0.00    0.00     0.00 mdi       *      SZ4KwU4
10.3.2.163            3          59   0    0.03    0.01     0.00 mdi       -      aaeQZDf
5

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

I formatted your post again.

Are you sure you did not define any tag for your AWS EC2 instances?
Could you double-check that?

6 
Name
Prod-MM-Jobs-ES3

PROJECT
PROD-JOBS

User
ec2-user

I have created above tags only. Now I am facing the problem is I have stopped elastic search service in master ec2 instance and automatically removed instance from the cluster and shard re allocation has been successfully completed.
Then started the elastic search but its not adding to the cluster again.

Again and stopped service from running master instance. Now the old instance added to the cluster.

7

According to the code:

If any tag is defined with discovery.ec2.tag.FOO: BAR, then AWS API will only send back instances which are having a tag FOO with value BAR.

Other instances should not come back and are going most likely not being able to join the cluster.

Can you reproduce your problem from scratch and describe here exactly all the steps?
Also sharing your logs would help.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.