Elastic ec2 plugin : access denied to aws

Elastic Stack Elasticsearch
1

Hi, I'm trying not to use access key and secret for aws api access and instead use ec2 instance profile to grant role based access.

I could not find text suggesting this is acceptable or not for elastic ec2 plugin to perform Discovery.

Appreciate if someone can shed some light.
We essentially cannot use access keys, as per security practices.

Thanks in anticipation.

2

Support for instance profile is in ec2 plugin. So it should work OOTB.

If you don't add the key and secret, we use then DefaultAWSCredentialsProviderChain to get that information. See this page: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html

3

Thank you, David. I still could not get the ec2-plugin to work.

On the node, I could run command aws ec2 describe-instances --region au-west-1 and can see the output (as user: root and ec2-user, with proxy setup and also no_proxy set to 169.254.169.254). Node is attached to instance profile.

However, elasticsearch logs show issues with ec2-plugin not able to perform discovery with following logs. Elastic, plugins, x-pack, kibana are .rpm packages.

[o.e.x.m.j.p.l.CppLogMessageHandler] [controller/29480] [Main.cc@128] controller (64 bit): Version 6.2.2 (Build 61c43548e5e0f2) Copyright (c) 2018 Elasticsearch BV
[o.e.d.DiscoveryModule    ] [10.10.10.10.my.domain] using discovery type [zen]
[o.e.n.Node               ] [10.10.10.10.my.domain] initialized
[o.e.n.Node               ] [10.10.10.10.my.domain] starting ...
[o.e.t.TransportService   ] [10.10.10.10.my.domain] publish_address {10.10.10.10:9300}, bound_addresses {127.0.0.1:9300}, {10.10.10.10:9300}
[o.e.b.BootstrapChecks    ] [10.10.10.10.my.domain] bound or publishing to a non-loopback address, enforcing bootstrap checks

**[o.e.d.e.AwsEc2UnicastHostsProvider] [10.10.10.10.my.domain] Exception while retrieving instance list from AWS API: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: 580a6081-fc13-4c88-8d89-12b76112c8ff)**

[o.e.c.s.MasterService    ] [10.10.10.10.my.domain] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {10.10.10.10.my.domain}{g66SeW8xQ1uUE7yz5GT6zQ}{WZk1XIi8QJWLKmJ2S7kbIQ}{10.10.10.10}{10.10.10.10:9300}{ml.machine_memory=7672758272, ml.max_open_jobs=20, ml.enabled=true}
[o.e.c.s.ClusterApplierService] [10.10.10.10.my.domain] new_master {10.10.10.10.my.domain}{g66SeW8xQ1uUE7yz5GT6zQ}{WZk1XIi8QJWLKmJ2S7kbIQ}{10.10.10.10}{10.10.10.10:9300}{ml.machine_memory=7672758272, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {10.10.10.10.my.domain}{g66SeW8xQ1uUE7yz5GT6zQ}{WZk1XIi8QJWLKmJ2S7kbIQ}{10.10.10.10}{10.10.10.10:9300}{ml.machine_memory=7672758272, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[o.e.h.n.Netty4HttpServerTransport] [10.10.10.10.my.domain] publish_address {10.10.10.10:9200}, bound_addresses{127.0.0.1:9200}, {10.10.10.10:9200

/etc/elasticsearch/elasticsearch.yml is as follows

cluster.name: elasticsearch-nonprod
node.name: ${HOSTNAME}
node.master: true
node.data: true
node.ingest: true
xpack.security.enabled: false
xpack.monitoring.enabled: true
path:
    logs: /var/log/elasticsearch
    data: /var/lib/elasticsearch

network.host: [_site_,_local_]
plugin.mandatory: discovery-ec2

discovery:
    zen.hosts_provider: ec2
    ec2:
        groups: sg-5b561000
        host_type: private_ip
        tag.Cluster: elasticsearch-nonprod
        availability_zones: eu-west-1a,eu-west-1b,eu-west-1c
        protocol: http
        proxy:
            host: 10.10.20.20
            port: 8080
4

Please format your code, logs or configuration files using </> icon as explained in this guide and not the citation button. It will make your post more readable.

Or use markdown style like:

```
CODE
```

There's a live preview panel for exactly this reasons.

Lots of people read these forums, and many of them will simply skip over a post that is difficult to read, because it's just too large an investment of their time to try and follow a wall of badly formatted text.
If your goal is to get an answer to your questions, it's in your interest to make it as easy to read and understand as possible.
Please update your post.

Could you also share how you create the repository? The exact command.
I think I know what is happening but I need that information first.

5

I'll format the text from laptop soon, thanks for the reminder.

But sorry I could understand which repository are you referring to ?
I downloaded the offline rpms, if that's the context.

Thanks for helping.

6

Ha my bad. I was thinking of repository-s3 plugin but you are using discovery-ec2. Sorry for the confusion.

So here you are missing the discovery.ec2.endpoint: ec2.eu-west-1.amazonaws.com setting I think.

See https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/_settings.html

7

That was the missing bit, works now!

Btw, how do I know which properties are optional. The documentation says endpoint is automatically derived from metadata, and the reason I skipped to add it in config.

Thanks a ton.

8

That's a documentation bug:

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.