Function could not deploy, error: bucket 'abc' already exist and you don't have permission to access it

Hi, I'm having the same issue as Unable to deploy functionbeat to s3 bucket, using Function Beat 7.1.1 Linux x86_64.

I do have full access to the bucket, and it does exist:

aws s3 ls s3://

I am logged in as root, so let me know if that's a problem. I'm running on EC2 with Role that grants me access to bucket.

Functionbeat.yml (slightly redacted):
functionbeat.provider.aws.deploy_bucket: ""
functionbeat.provider.aws.functions:

  • name: cloudwatch
    enabled: true
    type: cloudwatch_logs
    description: "lambda function for cloudwatch logs"
    env: staging
    triggers:
    • log_group_name: /aws/lambda/
  • name: sqs
    enabled: false
    type: sqs
    description: "lambda function for SQS events"
    triggers:
    • event_source_arn: arn:aws:sqs:us-east-1:xxxxx:myevents
  • name: kinesis
    enabled: false
    type: kinesis
    description: "lambda function for Kinesis events"
    triggers:
    • event_source_arn: arn:aws:sqs:us-east-1:xxxxx:myevents
      name: abcdef-funcbeat
      tags: ["", ""]
      env: staging
      queue:
      mem:
      events: 4096
      flush.min_events: 2048
      flush.timeout: 1s
      output.elasticsearch:
      hosts: ["ip-.us-east-2.compute.internal:9200"]
      setup.template.settings:
      setup.kibana:
      host: "http://:5601"
      logging.level: debug
      logging.to_files: true
      logging.files:

Please help! Thank you in advance!
David

Functionbeat checks if the bucket exists using HEAD bucket call. Unfortunately, this part of the code does not return the error exactly. I am opening a PR to fix it.

Have you set the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_DEFAULT_REGION correctly?

Ahh, I think I understand. I'm using the credentials provider (by default) but not the default profile, so I think I just need to set AWS_PROFILE. We don't have keys because I'm running on EC2 with an instance profile. I'll respond back when I test.

Nope, I was wrong. The instance profile is the default. I can do a aws s3 ls s3://<bucketname> and it works just fine.

Does the functionbeat not work with the default credential provider chain?

I see. Right now we don't support AWS profiles. Do you mind opening a feature request on GH? https://github.com/elastic/beats/issues/new?template=feature-request.md

I've made the request. I've maybe figured out a way around, but now I'm getting an error when the deploy executes cloudformation to create a role. We're not allowed to create roles. Can I specify a role instead?

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.