Azure append Blobs

Hi,
i'm new here.

I've an problem to read from an azure append blob file with filebeat.

i use this configuration:

 - type: azure-blob-storage
          id: my-azureblobstorage-id
          enabled: true
          account_name: ""
          auth.shared_credentials.account_key: ""
          containers:
          - name: commerce-logs-separated
            max_workers: 3
            poll: true
            poll_interval: 10s

But the message i've got is:

content-type application/octet-stream not supported

Thanks in advance for help.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Two problems:

  1. We do not support Append-type blobs with Filebeat, each time an append occurs we will treat it as if it was a new file and process the entire file again. See my recent comment on another person's post I want to read an append-format Blob file with filebeat - #2 by strawgate
  2. The content-type on the content you've uploaded to Azure blob storage is incorrect and there is no way to override it in the beat. The uploader sets the content-type when they write the blob and when we read the blob we trust the content-type to be correct. We really should have an option for overriding this so ive filed an issue in the Beats repo for this. [filebeat][azure-blob-storage] Support custom content-type · Issue #42971 · elastic/beats · GitHub

@lars-s, the issue with application/octet-stream not supported has been solved with later updates, the issue with append blobs is separate as mentioned by @strawgate.