Hello All,
Is there a way to throttle messages sent to elastic and indexing?
I saw a document on how to increase indexing speed.
Is there a way to decrease the index rate?
For example, if elastic gets 1 million messages per second, it will crash. Is there a way to digest these million messages slowly and surely?
Thanks!
When overloaded Elasticsearch will start rejecting requests and it is up to the client sending requests to handle this and back off. There is no other rate limiting functionality built in as far as I know.
Elastic will reject messages? Foe sure?
Is there a way to know what is the index rate elastic is capable of?
Thanks!
Yes, Elasticsearch has internal queues that are limited in size. Once these get exceeded (search or indexing requests), requests will be rejected.
That you will need to test because it depends on a large number of factors, e.g. index settings, hardware, data, mappings, sharding setup and the bulk size used.
This is set by the client, right?
Any settings on elastic side I can configure?
Thanks
Yes.
Not really. The defaults are generally quite good. You can tune queue sizes, but that is an advanced feature and can lead to a greater risk of the nodes falling over.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.