SNS Output - Logstash in EKS - using fine grained Service account IAM roles

[chandra2037 (Chandra)]

Elasticsearch Version: 7.4.0
Logstash Version: 7.4.0

Deployed in Amazon EKS as a POD using official Logstash Docker container.

Trying to publish messages to Amazon SNS using Logstash SNS output plugin.

Amazon introduced Fine-Grained IAM Roles for EKS Service Accounts to give fine-grained IAM role to Pods rather than EKS nodes.

Created IAM role with SNS:Publish access, and also service account with this new role.

Providing the service account details to the Logstash deployment manifest file. After deployment, I am seeing aws-iam-token volume which contains the AWS_WEB_IDENTITY_TOKEN_FILE, and also AWS_IAM_ROLE_ARN as environment variable.

Now coming to configuring SNS output, I am seeing following options to configure the access to publish messages to SNS

• access_key_id
• aws_credentials_file
• secret_access_key
• session_token

Since I have token file, I configured the session_token to point to the AWS_WEB_IDENTITY_TOKEN_FILE location. Also, I tried configuring AWS_IAM_ROLE_ARN as access_key_id.

But Logstash not able to publish messages to SNS, it is giving permission error. By default, it is assuming the EKS node role, and node role do not have access to publish messages to SNS. Configuring the session_token having no impact.

Am I missing something in the Output configuration?

Also noticed the following in the AWS documentation, not sure SDK version has any impact on this?

Thank you for any input on this issue.

[yaauie (Ry Biesemeyer)]

It looks like the current release of the SNS Output Plugin does not have a feature to read credentials from a file; both session_token and secret_access_key expect the key provided to be a literal string (not a file path), and aws_credentials_file is merely a file-based way of providing these two as a key/value file in a specific format.

Additionally, Logstash 7.4.0 ships with AWS SDK 2.11.343, so it does not support the sts:AssumeRoleWithWebIdentity credential provider:

    aws-sdk (2.11.343)

-- Gemfile.jruby-2.5.lock.release:45@v7.4.0

---

Right now the primary blocker to upgrading the AWS SDK that is shipped with these plugins is the sheer number of plugins that rely on the SDK requires that all are updated in lock-step to avoid dependency conflicts. We have work in-flight that mitigates this by distributing related plugins as a single "integration" plugin from a shared code-base, and regrouping the AWS plugins is on our to-do list.