# Functionbeat deploy fails, citing issue with s3 bucket **URL:** \ **Category:** Beats\ **Tags:** functionbeat\ **Created:** [January 11, 2021, 8:51pm UTC](https://discuss.elastic.co/t/functionbeat-deploy-fails-citing-issue-with-s3-bucket/260767 "2021-01-11T20:51:03Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Jonathan\_Detert](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonathan_detert/32/80849_2.png) [@Jonathan\_Detert](https://discuss.elastic.co/u/Jonathan_Detert)\ **Post date:** [January 11, 2021, 8:51pm UTC](https://discuss.elastic.co/t/functionbeat-deploy-fails-citing-issue-with-s3-bucket/260767/1 "2021-01-11T20:51:03Z")
I'm trying to use functionbeat for my first time, with a single function, of type kinesis, using functionbeat v6.8.13. When I run 'functionbeat deploy kinesis' (where 'kinesis' is the name i chose for the function name), I get an error "'enterprise-elk-dev-s3bucket' already exist and you don't have permission to access it Fail to deploy 1 function(s)" However, I can list, put, and rm objects in the bucket named above using the 'aws s3' cli. Any ideas what is wrong? Here's the contents of my functionbeat.yml: functionbeat.provider.aws.deploy\_bucket: "enterprise-elk-dev-s3bucket" functionbeat.provider.aws.functions: - name: kinesis enabled: true type: kinesis description: "lambda function for Kinesis events" triggers: - event\_source\_arn: arn:aws:kinesis:us-east-1:xxxxxxxxxxxx:stream/kinesis-stream-for-elk name: lambda2kinesis2es setup.kibana: output.elasticsearch: hosts: ["[https://elkdev-ingest.nml.com:443](https://elkdev-ingest.nml.com:443)"] username: "elastic" password: "${ES\_PWD}" ssl: verification\_mode: "none" processors: - add\_host\_metadata: ~ - add\_cloud\_metadata: ~ And here is the command I tried, it's output, and some more commands that show that the computer from which I ran the command, has permission to put files in the bucket: ./functionbeat -e deploy kinesis 2021-01-11T20:40:01.053Z INFO instance/beat.go:611 Home path: [/home/azscomp/functionbeat-6.8.13-linux-x86\_64] Config path: [/home/azscomp/functionbeat-6.8.13-linux-x86\_64] Data path: [/home/azscomp/functionbeat-6.8.13-linux-x86\_64/data] Logs path: [/home/azscomp/functionbeat-6.8.13-linux-x86\_64/logs] 2021-01-11T20:40:01.053Z INFO instance/beat.go:618 Beat UUID: f331a229-6c10-45b2-82c4-068775f0e363 Function: kinesis, could not deploy, error: bucket 'enterprise-elk-dev-s3bucket' already exist and you don't have permission to access it Fail to deploy 1 function(s) aws s3 ls s3://enterprise-elk-dev-s3bucket /usr/lib/python2.7/site-packages/awscli/customizations/cloudfront.py:17: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.hazmat.primitives import serialization, hashes aws s3 cp package.zip s3://enterprise-elk-dev-s3bucket/package.zip /usr/lib/python2.7/site-packages/awscli/customizations/cloudfront.py:17: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.hazmat.primitives import serialization, hashes upload: ./package.zip to s3://enterprise-elk-dev-s3bucket/package.zip aws s3 ls s3://enterprise-elk-dev-s3bucket /usr/lib/python2.7/site-packages/awscli/customizations/cloudfront.py:17: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.hazmat.primitives import serialization, hashes 2021-01-11 20:40:30 15263071 package.zip aws s3 rm s3://enterprise-elk-dev-s3bucket/package.zip /usr/lib/python2.7/site-packages/awscli/customizations/cloudfront.py:17: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release. from cryptography.hazmat.primitives import serialization, hashes delete: s3://enterprise-elk-dev-s3bucket/package.zip So, any idea why I can't put objects in the s3 bucket? Also, I tried modifying the name of the bucket in the functionbeat.yml to a nonexistent bucket, and then rerunning the functionbeat setup command, and it fails with the same error, which says that the bucket exists, but in this test, the bucket did not exist. So, I don't know that the error output can be trusted. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [January 12, 2021, 7:09pm UTC](https://discuss.elastic.co/t/functionbeat-deploy-fails-citing-issue-with-s3-bucket/260767/2 "2021-01-12T19:09:16Z")
Hey @Jonathan_Detert, I was looking at the code that returns this error ([here](https://github.com/elastic/beats/blob/f9df2493f9b757f7d5eeb0df8d11769f23949eb8/x-pack/functionbeat/manager/aws/op_ensure_bucket.go#L58)), and it seems to be a catch-all error when trying to check for the existence of the bucket. For all cases, except when the bucket doesn't exist, it returns the same error. For example, if authentication is failing, Functionbeat is going to return the same error for any bucket name. Could you review the credentials you are using? If you have the chance of compiling Functionbeat yourself, you can try to modify this code to report the exact error that is happening there. I have opened an issue to improve this: [https://github.com/elastic/beats/issues/23466](https://github.com/elastic/beats/issues/23466) ---
**Author:** ![Jonathan\_Detert](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonathan_detert/32/80849_2.png) [@Jonathan\_Detert](https://discuss.elastic.co/u/Jonathan_Detert)\ **Post date:** [January 12, 2021, 7:18pm UTC](https://discuss.elastic.co/t/functionbeat-deploy-fails-citing-issue-with-s3-bucket/260767/3 "2021-01-12T19:18:45Z")
Hi @jsoriano, thanks for looking at this. I think that the error message should not assert a lie. I think it can say "either it doesn't exist, or you don't have access to it". I ran the code from an ec2 instance that has assumed a role that grants read/write/list of the s3 bucket. w.r.t. authN/authZ, the [documention](https://www.elastic.co/guide/en/beats/functionbeat/6.8/functionbeat-deploying.html#functionbeat-deploying) says that 'you _ **can** _ set environment variables that contain your credentials'. It doesn't say that you must. I did not, because I was relying on the ec2 instance having assumed a role w. the necessary authZ. Are the env vars the only way? If so, it should be updated to make that clear. Also, that would be a problem for me, as I am prohibited by my employer from obtaining an aws access key. ---
**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\ **Post date:** [January 12, 2021, 7:36pm UTC](https://discuss.elastic.co/t/functionbeat-deploy-fails-citing-issue-with-s3-bucket/260767/4 "2021-01-12T19:36:59Z")
> [@Jonathan\_Detert](#): > > Are the env vars the only way? If so, it should be updated to make that clear. Also, that would be a problem for me, as I am prohibited by my employer from obtaining an aws access key. It seems that the docs have been recently updated, there are more details now: [Configure AWS functions | Functionbeat Reference [7.16] | Elastic](https://www.elastic.co/guide/en/beats/functionbeat/7.x/configuration-functionbeat-options.html#aws-credentials-config) It says now: The `aws` functions require AWS credentials configuration in order to make AWS API calls. Users can either use `AWS_ACCESS_KEY_ID` , `AWS_SECRET_ACCESS_KEY` and/or `AWS_SESSION_TOKEN` , or use shared AWS credentials file. Please see [AWS credentials options](https://www.elastic.co/guide/en/beats/functionbeat/7.x/configuration-functionbeat-options.html#aws-credentials-config) for more details. So it seems that you need to use environment variables, or a credentials file. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 9, 2021, 9:37pm UTC](https://discuss.elastic.co/t/functionbeat-deploy-fails-citing-issue-with-s3-bucket/260767/5 "2021-02-09T21:37:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.