# Cloudflare Logpush integration

**URL:** <https://discuss.elastic.co/t/cloudflare-logpush-integration/364437>\
**Category:** Elastic Agent\
**Tags:** integrations\
**Created:** [August 6, 2024, 7:09am UTC](https://discuss.elastic.co/t/cloudflare-logpush-integration/364437 "2024-08-06T07:09:58Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alessio\_Creo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alessio_creo/32/43381_2.png) [@Alessio\_Creo](https://discuss.elastic.co/u/Alessio_Creo)\
**Post date:** [August 6, 2024, 7:09am UTC](https://discuss.elastic.co/t/cloudflare-logpush-integration/364437/1 "2024-08-06T07:09:58Z")

</div>

Hi,

we're ingesting data from MinIO bucket using Cloudflare logpush integrations, due to the huge amount of data sometimes the data flow is not "real time" because a single agent can handle the data, using more than one agent seems like not collecting data at the same pace.

Is there any way to parallelize or optimize data ingestion?

Thanks,  
Alessio

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [August 6, 2024, 6:19pm UTC](https://discuss.elastic.co/t/cloudflare-logpush-integration/364437/2 "2024-08-06T18:19:52Z")

</div>

> [@Alessio\_Creo](#):
>
> we're ingesting data from MinIO bucket using Cloudflare logpush integrations

Are you using the Cloudflare Logpush integration with the S3 polling mode, and configured it to poll your bucket on MinIO, right?

The main issue with the polling mode is that it does not scale and you cannot parallelize it.

Adding another agent would probablly lead to duplicate data.

To have multiple agents consuming this data it is required to use SQS, but I don't think that this is possible since you are using MinIO and not AWS.

Is that just one site or you have multiple sites sending logs to the same bucket?

If you have multiple sites sending log to the same bucket you could create one bucket per site, and then use one integration per bucket.

---

<div class="post-metadata">

**Author:** ![Alessio\_Creo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alessio_creo/32/43381_2.png) [@Alessio\_Creo](https://discuss.elastic.co/u/Alessio_Creo)\
**Post date:** [August 29, 2024, 9:16am UTC](https://discuss.elastic.co/t/cloudflare-logpush-integration/364437/3 "2024-08-29T09:16:02Z")

</div>

Sorry for the late answer! Thank you very much!