AWS IAM role not working with Elasticsearch 5.0.0-alpha2, but works with 2.3 version

AWS IAM role not working with Elasticsearch 5.0.0-alpha2, but works with 2.3 version

I am trying to use AWS IAM role with Elasticsearch 5.0.0-alpha2 and EC2 discovery plug-in, but it does not seem to be working and I am getting below error:

"Exception while retrieving instance list from AWS API: Authorization header or parameters are not formatted correctly. (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure"

I am using below configuration with jdk8: "test-cluster" "us-west-2" "us-west-2" "http"
discovery.type: "ec2"
#bootstrap.mlockall: true
node.master: true false ${HOSTNAME}-Master
discovery.zen.minimum_master_nodes: 1 ec2:privateIp
discovery.ec2.any_group: true
discovery.ec2.groups : sg-9d856tfe

However, same configuration works fine with with Elasticsearch 2.3 version. Has anyone faced the same issue with the latest Elasticsearch version 5.0.0-alpha2?

Thanks for reporting! That's weird.

Is discovery.ec2.groups : really like this (empty I mean)?

What if you remove those 2 lines?

discovery.ec2.any_group: true
discovery.ec2.groups :

If you can narrow down what is the setting which is causing the issue, that would help a lot!

Thanks for response David. discovery.ec2.groups is not empty. It is security group id e.g. sg-9d856tfe.

I am using security group for clustering so that instances belonging to same security group can be clustered. For your information, same configuration works if I provide access_key and secret_access_key.

Therefore, I feel problem happens when plugin is trying to fetch instance list while making AWS API call and its not using IAM role for this purpose. As per documentation, plug-in always tries to use IAM role first and if IAM role is not available, then only it should look for access_key and secret_access_key and my elasticsearch instance is associated with IAM role with correct permission to fetch the instance list. This works fine 2.3 version with IAM role.

As you suggested, I will remove below lines and try - but I am not sure if it will solve the problem that I am facing.
discovery.ec2.any_group: true
discovery.ec2.groups : sg-9d856tfe

Please let me know, if you need more info. Thanks!

Ok. Thanks for the clarification.

Could you open an issue in elasticsearch repo please with all those details?

Sure David, will do that. Thanks.

Hi David,

As you said, I have opened an issue in elasticsearch repo and It was indeed a bug. Below is the issue link:

Seems it got fixed now. However, I am not able to test this. I have removed the discovery-ec2 plug-in and installed again(as shown below), but getting the same error.

bin/elasticsearch-plugin remove discovery-ec2
bin/elasticsearch-plugin install discovery-ec2

will it not download the latest code, if I install the discovery-ec2 plug-in again after removing the old one? Can you please help me here? Thanks.