We are planning to send logs from Boomi to Elasticsearch directly using ELK REST APIs without using either Beats or Logstash.
Document APIs | Elasticsearch Guide [8.2] | Elastic
Is there any rate limiting for these APIs
Elasticsearch does not in itself enforce any rate limiting, but there are limits to what a cluster can handle. Ensure you use bulk requests though as adding documents one at a time adds a LOT of overhead and limits indexing speed.
If you are sending logs directly to Elasticsearch instead of writing then to file and having filebeat collect them from there, consider what impact Elasticsearch being temporarily unavailable would have on your application and log collection.
2 Likes
Thanks a lot for the quick reply.
We have a JMS queue in between Dell Boomi and Elasticsearch and we are using Elastic cloud. So, even if it fails to send the data to Elastic Cloud, it go back to the JMS Queue. But, the load on cluster is a concern
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.