# A huge number of Listblobs commands are being executed from filebeat to Azure Storage

**URL:** <https://discuss.elastic.co/t/a-huge-number-of-listblobs-commands-are-being-executed-from-filebeat-to-azure-storage/359772>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 20, 2024, 3:17am UTC](https://discuss.elastic.co/t/a-huge-number-of-listblobs-commands-are-being-executed-from-filebeat-to-azure-storage/359772 "2024-05-20T03:17:48Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [May 21, 2024, 2:09pm UTC](https://discuss.elastic.co/t/a-huge-number-of-listblobs-commands-are-being-executed-from-filebeat-to-azure-storage/359772/3 "2024-05-21T14:09:24Z")

</div>

Hello and welcome,

> [@syo04suke26](#):
>
> Is my assumption above (filebeat executes the ListBlobs command as many times as there are blob files) correct as a specification of filebeat?

If I'm not wrong, yes, this is how it works, it will poll the storage container each poll interval and list the files in the container to know the files it needs to download and process.

It is mentioned in the [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-azure-blob-storage.html#filebeat-input-azure-blob-storage) that this can get expensive if you have a large number of files.

> Polling is generally recommented for most cases even though **it can get expensive with dealing with a very large number of files**.

> [@syo04suke26](#):
>
> Is there anything I can do to avoid executing a large number of ListBlobs commands?

I don't think so, this is how polling work when consuming data from blob storage, this is the same thing as polling on S3 or GCS, with large buckets it can get expensive.

One alternative would be to change your data pipeline and put your logs inside an Event Hub instead of storage containers.

---

_[View the full topic](https://discuss.elastic.co/t/a-huge-number-of-listblobs-commands-are-being-executed-from-filebeat-to-azure-storage/359772)._
