AWS IAM role not working with Elasticsearch 5.0.0-alpha2, but works with 2.3 version

Elastic Stack Elasticsearch
1

AWS IAM role not working with Elasticsearch 5.0.0-alpha2, but works with 2.3 version

I am trying to use AWS IAM role with Elasticsearch 5.0.0-alpha2 and EC2 discovery plug-in, but it does not seem to be working and I am getting below error:

"Exception while retrieving instance list from AWS API: Authorization header or parameters are not formatted correctly. (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure"

I am using below configuration with jdk8:

cluster.name: "test-cluster"
cloud.aws.region: "us-west-2"
cloud.aws.ec2.region: "us-west-2"
cloud.aws.ec2.protocol: "http"
discovery.type: "ec2"
#bootstrap.mlockall: true
node.master: true
node.data: false
node.name: ${HOSTNAME}-Master
discovery.zen.minimum_master_nodes: 1
network.host: ec2:privateIp
discovery.ec2.any_group: true
discovery.ec2.groups : sg-9d856tfe

However, same configuration works fine with with Elasticsearch 2.3 version. Has anyone faced the same issue with the latest Elasticsearch version 5.0.0-alpha2?

2

Thanks for reporting! That's weird.

Is discovery.ec2.groups : really like this (empty I mean)?

What if you remove those 2 lines?

discovery.ec2.any_group: true
discovery.ec2.groups :

If you can narrow down what is the setting which is causing the issue, that would help a lot!

3

Thanks for response David. discovery.ec2.groups is not empty. It is security group id e.g. sg-9d856tfe.

I am using security group for clustering so that instances belonging to same security group can be clustered. For your information, same configuration works if I provide access_key and secret_access_key.

Therefore, I feel problem happens when plugin is trying to fetch instance list while making AWS API call and its not using IAM role for this purpose. As per documentation, plug-in always tries to use IAM role first and if IAM role is not available, then only it should look for access_key and secret_access_key and my elasticsearch instance is associated with IAM role with correct permission to fetch the instance list. This works fine 2.3 version with IAM role.

As you suggested, I will remove below lines and try - but I am not sure if it will solve the problem that I am facing.
discovery.ec2.any_group: true
discovery.ec2.groups : sg-9d856tfe

Please let me know, if you need more info. Thanks!

4

Ok. Thanks for the clarification.

Could you open an issue in elasticsearch repo please with all those details?

5

Sure David, will do that. Thanks.

6

Hi David,

As you said, I have opened an issue in elasticsearch repo and It was indeed a bug. Below is the issue link:
https://github.com/elastic/elasticsearch/issues/18652

Seems it got fixed now. However, I am not able to test this. I have removed the discovery-ec2 plug-in and installed again(as shown below), but getting the same error.

bin/elasticsearch-plugin remove discovery-ec2
bin/elasticsearch-plugin install discovery-ec2

will it not download the latest code, if I install the discovery-ec2 plug-in again after removing the old one? Can you please help me here? Thanks.