# SQSSNSS3 plugin error queue not valid for endpoint **URL:** \ **Category:** Logstash\ **Created:** [August 27, 2021, 3:38pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685 "2021-08-27T15:38:27Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 9, 2021, 10:14pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/21 "2021-09-09T22:14:31Z")
@andrewkroh do you see anything in the logs above? Its saying the the queueURL is not in the correct format but I am not familiar with that particulars. It is in amazon Gov Cloud I believe is that an issue? You can see the config above... ```auto - type: aws-s3 queue_url: https://sqs.us-gov-east-redacted. role_arn: arn:aws-us-gov:iam::redacted expand_event_list_from_field: Records bucket_list_interval: 300s file_selectors: - regex: '/CloudTrail/' - regex: '/CloudTrail-Digest/' - regex: '/CloudTrail-Insight/' ``` ---
**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\ **Post date:** [September 9, 2021, 10:38pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/22 "2021-09-09T22:38:11Z")
> [@afoster](#): > > "queue\_url": "[https://sqs.us-gov-east-1.amazon.com/redacted](https://sqs.us-gov-east-1.amazon.com/redacted)"}` Is that a valid gov-cloud URL? I thought the domains were `amazonaws.com` not `amazon.com`. If it is infact the correct domain then set `endpoint: amazon.com`. ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 10, 2021, 2:21pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/23 "2021-09-10T14:21:40Z")
the SQS Queue is valid, I copied it from the AWS console. Can you please tell me where to set the endpoint: [amazon.com](http://amazon.com) is that in the filebeat input Can you confirm if this [Fix hardcoded region](https://github.com/elastic/beats/pull/24861) is resolved? I am on Filebeat 7.14 ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 10, 2021, 8:29pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/24 "2021-09-10T20:29:29Z")
Hi @afoster Yes that fix is int the codebase for 7.14 it was fixed in 7.12.1 see [here](https://www.elastic.co/guide/en/beats/libbeat/7.14/release-notes-7.12.1.html) - Fix S3 input validation for non [amazonaws.com](http://amazonaws.com) domains. [24420](https://github.com/elastic/beats/issues/24420) [24861](https://github.com/elastic/beats/pull/24861) Also I found [this discuss](https://discuss.elastic.co/t/filebeat-aws-module-s3-input-error-queueurl-is-not-in-format/261850/5) and this post by the person that found and opened that issues states it should not affect Gov Cloud see [here](https://discuss.elastic.co/t/filebeat-aws-module-s3-input-error-queueurl-is-not-in-format/261850/9) According to this > **[FIPS - Amazon Web Services (AWS)](https://aws.amazon.com/compliance/fips/)** So perhaps still just a config issue? So seeing this format is expected `https://sqs.{REGION_ENDPOINT}.{ENDPOINT}/{ACCOUNT_NUMBER}/{QUEUE_NAME}` Is the config in this format? ```auto - type: aws-s3 queue_url: sqs.us-gov-east-1.amazonaws.com/{ACCOUNT_NUMBER}/{QUEUE_NAME} ``` ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 10, 2021, 8:30pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/25 "2021-09-10T20:30:31Z")
I have actually confirmed it should have been amazonaws. I modified the config for the correct SQS Queue URL. However I am still getting invalid token errors. ```auto The security token included in the request is invalid status code: 403, request id: ea537a51-790c-4079-b9c2-7c6f180d2514 {"id": "F887DBA4DA00DEE3", "queue_url": "https://sqs.us-gov-east-1.amazonaws.com/redacted", "region": "us-gov-east-1"} 2021-09-10T20:26:19.465Z ERROR [input.aws-s3] awss3/collector.go:106 SQS ReceiveMessageRequest failed: InvalidClientTokenId: The security token included in the request is invalid status code: 403, request id: 1c06c489-6500-45d7-9fa3-0a0e4ccfc4d5 {"id": "F887DBA4DA00DEE3", "queue_url": "https://sqs.us-gov-east-1.amazonaws.com/redacted", "region": "us-gov-east-1"} 2021-09-10T20:26:19.478Z ERROR [input.aws-s3] awss3/collector.go:106 SQS ReceiveMessageRequest failed: InvalidClientTokenId: The security token included in the request is invalid status code: 403, request id: e54a0a86-57ef-4421-b471-f105eb45b37d {"id": "F887DBA4DA00DEE3", "queue_url": "https://sqs.us-gov-east-1.amazonaws.com/redacted", "region": "us-gov-east-1"} ``` ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 10, 2021, 8:31pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/26 "2021-09-10T20:31:30Z")
It is now. Thank you, Stephen. I incorrectly missed the aws in editing. I have corrected that to match your url and am seeing errors. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 10, 2021, 8:46pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/27 "2021-09-10T20:46:01Z")
Ok good.. I'm not as familiar with the authentication portion of but would definitely take a close look at the docs [here](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-aws.html#aws-credentials-options) I think you are close...just need to get the creds right. > The aws module requires AWS credentials configuration in order to make AWS API calls. Users can either use access\_key\_id, secret\_access\_key and/or session\_token, or use role\_arn AWS IAM role, or use shared AWS credentials file. > > Please see AWS credentials options for more details. ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 10, 2021, 8:59pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/28 "2021-09-10T20:59:34Z")
Thank you Stephen. I will look into this. Appreciate your assistance. ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 15, 2021, 8:19pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/29 "2021-09-15T20:19:07Z")
Hello, I resolved the token and the AWS input errors remain. any ideas is this the correct input? ```auto etc/filebeat/filebeat.yml --path.home /usr/share/filebeat --path> ERROR [input.aws-s3] awss3/collector.go:106 > Sep 15 19:42:27 ip-redacted filebeat[449]: status code: 403, request id: 7bef1c17-c967-4875-b1cd-df9e89cd14a1 {"id": "8448D> Sep 15 19:42:27 ip-redacted filebeat[449]: 2021-09-15T19:42:27.061Z ERROR [input.aws-s3] awss3/collector.go:106 > ``` ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 15, 2021, 10:13pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/30 "2021-09-15T22:13:38Z")
I also have noted that /var/log/syslog and /usr have filled up. I am not sure the reason they are full. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 16, 2021, 2:53pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/31 "2021-09-16T14:53:01Z")
Those error lines are incomplete / truncated so hard to tell can you provide a fuller error dump like you did above? but 403, It is still an auth issue.. What does your aws-s3 input look like now? please show as complete as possible and just anonymize the creds / queue name. Also is this Fips Enabled see [here](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-aws-s3.html#_fips_enabled) Also can you look at the SQS console and see any auth errors on that side. ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 16, 2021, 5:50pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/32 "2021-09-16T17:50:11Z")
Thank you Stephen. I will look into fips. I am not using aws module, and am using only filebeat keystore for credentials. ```auto filebeat.inputs: # Each - is an input. Most options can be set at the input level, so - type: aws-s3 queue_url: https://sqs.us-gov-east-1.amazonaws.com/redacted role_arn: arn:aws-us-gov:iam::redacted expand_event_list_from_field: Records bucket_list_interval: 300s file_selectors: - regex: '/CloudTrail/' - regex: '/CloudTrail-Digest/' - regex: '/CloudTrail-Insight/' # Change to true to enable this input configuration. enabled: true # filestream is an input for collecting log messages from files. It is going to replace log input in the future. - type: filestream enabled: false # ================================== Outputs =================================== # Configure what output to use when sending the data collected by the beat. output.logstash: hosts: ["localhost:5044"] output.elasticsearch.password: "$[ES_PWD}" ``` ```auto type or paste code here ``` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 16, 2021, 5:57pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/33 "2021-09-16T17:57:30Z")
If you put the credentials directly into the filebeat.yml does it work? Not sure What the reference to the module is. I understand you're using the aws-s3 input. ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 16, 2021, 6:00pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/34 "2021-09-16T18:00:14Z")
okay, just wanted to add for clarity what module I am using. great suggestion, Stephen. Ill do that straightaway and let you know the outcome. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 16, 2021, 6:02pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/35 "2021-09-16T18:02:30Z")
And when I'm speaking of credentials I'm speaking of the AWS S3 credentials such as access\_key\_id, secret\_access\_key ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 16, 2021, 6:02pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/36 "2021-09-16T18:02:48Z")
yes. understand. thank you ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 16, 2021, 6:04pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/37 "2021-09-16T18:04:02Z")
> [@afoster](#): > > ```auto > Configure what output to use when sending the data collected by the beat. > output.logstash: > hosts: ["localhost:5044"] > output.elasticsearch.password: "$[ES_PWD}" << Not needed > > ``` Also this is incorrect there is no reason to include the Elasticsearch password when you are sending The output to logstash. ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 16, 2021, 6:05pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/38 "2021-09-16T18:05:45Z")
ah. the documentation led me to believe this was required for the filebeat keystore. Apologies. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [September 16, 2021, 6:11pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/39 "2021-09-16T18:11:43Z")
Plus if you use the keystore you still need to put the fields in the yml example below, it does not automatically put them in the filebeat.yml but I would try first with them directly in the filebeat.yml Adding the keys to the keystore (it will prompt for the value) ```auto filebeat keystore add AWS_ACCESS_KEY_ID filebeat keystore add AWS_SECRET_ACCESS_KEY ``` then in the filebeat.yml ```auto - type: aws-s3 queue_url: https://sqs.us-gov-east-1.amazonaws.com/redacted role_arn: arn:aws-us-gov:iam::redacted expand_event_list_from_field: Records bucket_list_interval: 300s access_key_id: ${AWS_ACCESS_KEY_ID} secret_access_key: ${AWS_SECRET_ACCESS_KEY} file_selectors: - regex: '/CloudTrail/' - regex: '/CloudTrail-Digest/' - regex: '/CloudTrail-Insight/' # Change to true to enable this input configuration. enabled: true ``` ---
**Author:** ![afoster](https://avatars.discourse-cdn.com/v4/letter/a/9fc348/32.png) [@afoster](https://discuss.elastic.co/u/afoster)\ **Post date:** [September 16, 2021, 6:14pm UTC](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685/40 "2021-09-16T18:14:13Z")
> [@stephenb](#): > > ```auto > filebeat keystore add AWS_ACCESS_KEY_ID > > filebeat keystore add AWS_SECRET_ACCESS_KEY > > ``` thank you very much Stephen. much appreciated. [Previous page](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685.md?page=1) [Next page](https://discuss.elastic.co/t/sqssnss3-plugin-error-queue-not-valid-for-endpoint/282685.md?page=3)