Using the CludFlare integration, in case of Elastic failure, will the logs that have been sent to elastic be lost?
Or there will be queues (or something similar) from which Elastic will restart once it is reset.
Or alternatively will it recover the lost data directly from CloudFlare?
What do u mean?? Can u provide context??
The underlying Beats themselves have internal tracking and queues to store data if they cannot contact Elasticsearch.
How Filebeat works | Filebeat Reference [8.4] | Elastic is one example of how that works.
I have to push/pull Cloudflare logs to an Elastic Cloud.
So I thought to use the Cloudflare integration.
So in case this solution worked, my doubts were about data loss.
considering that the hypothetical architecture will be: Cloudflare <--> Elastic Agent (integration Cloudflare) <--> Elastic Cloud. Do you think it is possible and that your answer also applies to this architecture?
So it's been found that the current integration using the logpull api can fall behind due to the extreme amount of data cloudflare produces. It may fall behind but shouldn't lose/drop data but I can't be 100%. The log push api long term should be more effective.