Function could not deploy, error: bucket 'abc' already exist and you don't have permission to access it

Elastic Stack Beats
1

Hi, I'm having the same issue as Unable to deploy functionbeat to s3 bucket, using Function Beat 7.1.1 Linux x86_64.

I do have full access to the bucket, and it does exist:

aws s3 ls s3://

I am logged in as root, so let me know if that's a problem. I'm running on EC2 with Role that grants me access to bucket.

Functionbeat.yml (slightly redacted):
functionbeat.provider.aws.deploy_bucket: ""
functionbeat.provider.aws.functions:

  • name: cloudwatch
    enabled: true
    type: cloudwatch_logs
    description: "lambda function for cloudwatch logs"
    env: staging
    triggers:
    • log_group_name: /aws/lambda/
  • name: sqs
    enabled: false
    type: sqs
    description: "lambda function for SQS events"
    triggers:
    • event_source_arn: arn:aws:sqs:us-east-1:xxxxx:myevents
  • name: kinesis
    enabled: false
    type: kinesis
    description: "lambda function for Kinesis events"
    triggers:
    • event_source_arn: arn:aws:sqs:us-east-1:xxxxx:myevents
      name: abcdef-funcbeat
      tags: ["", ""]
      env: staging
      queue:
      mem:
      events: 4096
      flush.min_events: 2048
      flush.timeout: 1s
      output.elasticsearch:
      hosts: ["ip-.us-east-2.compute.internal:9200"]
      setup.template.settings:
      setup.kibana:
      host: "http://:5601"
      logging.level: debug
      logging.to_files: true
      logging.files:

Please help! Thank you in advance!
David

2

Functionbeat checks if the bucket exists using HEAD bucket call. Unfortunately, this part of the code does not return the error exactly. I am opening a PR to fix it.

Have you set the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION correctly?

3

Ahh, I think I understand. I'm using the credentials provider (by default) but not the default profile, so I think I just need to set AWS_PROFILE. We don't have keys because I'm running on EC2 with an instance profile. I'll respond back when I test.

4

Nope, I was wrong. The instance profile is the default. I can do a aws s3 ls s3://<bucketname> and it works just fine.

Does the functionbeat not work with the default credential provider chain?

5

I see. Right now we don't support AWS profiles. Do you mind opening a feature request on GH? https://github.com/elastic/beats/issues/new?template=feature-request.md

6

I've made the request. I've maybe figured out a way around, but now I'm getting an error when the deploy executes cloudformation to create a role. We're not allowed to create roles. Can I specify a role instead?

7

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.