Filebeat Cisco Umbrella

Elastic Stack Beats
1

I'm trying to setup Filebeat to use the Cisco Umbrella module. We have the Cisco maintained bucket. The URL Cisco provides is s3://cisco-managed-us-west-1/2613934_1b0f4f029c8f0b75a2f9a6d4e06a79d6cbbc41bb

The error I'm getting is

"ERROR [input.s3] s3/input.go:93 getRegionFromQueueURL failed: queueURL is not in format: https://sqs.{REGION_ENDPOINT}.amazonaws.com/{ACCOUNT_NUMBER}/{QUEUE_NAME} {"queue_url": "s3://cisco-managed-us-west-1/2613934_1b0f4f029c8f0b75a2f9a6d4e06a79d6cbbc41bb"}"

My module config is

umbrella:
   enabled: true
   var.input: s3
   # AWS SQS queue url
   var.queue_url: s3://cisco-managed-us-west-1/2613934_1b0f4f029c8f0b75a2f9a6d4e06a79d6cbbc41bb
   # Access ID to authenticate with the S3 input
   var.access_key_id: <mykey>
   # Access key to authenticate with the S3 input
   var.secret_access_key: <mypass>
   # The duration that the received messages are hidden from ReceiveMessage request
   #var.visibility_timeout: 300s
   # Maximum duration before AWS API request will be interrupted
   #var.api_timeout: 120s

The Cisco module documentation says that you can use the Cisco managed bucket, but I'm just not sure of the format of the var.queue_url. Does anyone know the trick?

2

Hello! You would need to set up an SQS for notification when there's new logs send into S3. Please see https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-s3.html#_s3_and_sqs_setup for more info. Thanks!

3

Hi @nathanachey! Unfortunately we do not support Cisco-managed S3 buckets at this time and have updated our docs accordingly. Cisco-managed buckets do not leverage SQS notifications, but Filebeat's S3 input relies on SQS. We are exploring possible solutions with a view to supporting Cisco-managed buckets in the future.

1 Like
4

OK

Thanks for the clarification. I will look into get our own S3 bucket.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.