Azure-blob-storage input: fatal error: concurrent map iteration and map write

I am getting an error using filebeat with azure-blob-storage input plugin. I am using filebeat version 8.8.1 and the following configuration:

- type: azure-blob-storage
  id: <id>
  enabled: true
  account_name: <account_name>
  max_workers: 1
  poll: true
  poll_interval: 15s
  auth.shared_credentials.account_key: <account_key>
  - name: <container_name_1>
  - name: <container_name_2>
  - name: <container_name_3>

Filebeats panic after a few minutes with the following log:

fatal error: concurrent map iteration and map write

goroutine 75 [running]:{{0x558f5d84a960?, 0xc0073fac00?}}, 0xc0005bddd0) +0xac, {0x558f5d188840?, 0xc0005bddd0?}) +0x3d, {0x558f5d188840?, 0xc0005bde28?, 0x558f5f8121a0?}) +0xda, {0x558f5d43d800?, 0xc0005bde00?, 0x558f59c892e4?}) +0xa9, {0x558f5d43d800?, 0xc0005bde00?, 0x16?}) +0x89, {0x558f5d43d800?, 0xc0005bde00?, 0xc00461d8a0?}) +0x83, {0x558f5ceddb80?, 0xc0005bde00?, 0x558f59366234?}) +0x115, {0x558f5ceddb80?, 0xc0005bde00?, 0xc00461d970?}) +0x9d, {0x558f5ceddb80?, 0xc0005bde00}) +0x27f*Iterator).Fold(...)*Converter).Convert(0xc004be38e0, {0x558f5cefc140, 0xc0000ef558}, {0x558f5ceddb80, 0xc0005bde00}) +0x13a{0x558f5cefc140, 0xc0000ef558}, {0x558f5ceddb80, 0xc0005bde00}) +0xcb*updateOp).Execute(0xc007248840, 0x0?) +0x199, {0xc00021e0a0?, 0x1c?, 0x0?}) +0x9f*eventDataACKer).onACK(0xc00059e1c0, 0x558f593974a0?, 0x1c) +0x134*trackingACKer).ACKEvents(0xc0006541e0, 0x1c) +0x203*bufferingEventLoop).processACK.func1() +0x24*bufferingEventLoop).processACK(0xc000e9a5a0, {0x0, 0x0}, 0x800) +0x1c5*ackLoop).handleBatchSig(0xc000e8aa60) +0xa4*ackLoop).run(0xc000e8aa60) +0x192 +0x65
created by +0x52d


I have tried to apply the same changes addressing the concurrency issue to the azure-blob-storage input plugin which were applied to the gcs plugin:

[filebeat][gcs] - Added missing locks for safe concurrency by ShourieG · Pull Request #34914 · elastic/beats (

[8.8](backport #35605) [filebeat][gcs] Fix for concurrency issues and context timeouts in the GCS input by mergify[bot] · Pull Request #35666 · elastic/beats (

But the issue still persists.

1 Like

Hi @kdobmayer,
Thanks for bringing this up.
There are some concurrency issues present in the azure blob storage input atm. There are fixes planned for this here Improve Azure blob storage input · Issue #35126 · elastic/beats · GitHub . However this is due for 8.11.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.