Filebeat S3 Input - Output Garbled

I have configured VPC Flow logs to ship to S3 and then an SQS message queue to notify Filebeat.
This creates records in ES but they appear like this! Any ideas?

Hi @jbws, thanks for giving this a try! What version of Filebeat are you using?

Version is 7.4.2 running in a container.


    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

  - type: s3
    queue_url: zzz
    access_key_id: aaa
    secret_access_key: bbb
    visibility_timeout: 1200 yyy
cloud.auth: xxx


setup.ilm.enabled: false

Does the s3 input plugin automatically unzip .gz files?


Thanks for the info! Yeah it's because VPC logs are gz files. added the support and will be released in 7.5 Filebeat.

Also I just want to point out, we have a ticket to add a VPC fileset in Filebeat as well:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.