Read elastic rate limit headers from response

Elastic Stack Elasticsearch
1

Hi,,

we are using provided code

const  response = await client.app.indexDocuments({
      engine_name: engineName,
      documents: snakeCaseDocuments,
    });

We are hitting API requests limits and we would like to implement debouncing for our request but we can not get the limits.

We have added 200ms debounce but it slows down our requests a lot we would like to track the limits and react based on that.

Thank you

Best regards

2

What is the specification of your cluster?

If you are looking to optimise indexing throughput I would recommend you follow these guidelines and use bulk requests as indexing individual documents adds a lot of overhead.

3

Is there a reason why index documents in the background doesnt use bulk insert? Doest it do one request per document then i didnt see the implementation of the function?

And this is our setup but we are looking into serverless since we would like a better managed scaling. At least we think that will help us since we have thousands of users that will be using elastic to sync and search their data.

image
image1408×692 76 KB

4

We are also not sure which type of hardware profile should be best for us since we have a high load on indexing

image
image854×155 7.77 KB

5

Hi Cristian,

also index documents shouldnt index documents by individiual documents.

    indexDocuments(this: That, params: T.IndexDocumentsRequest, options?: TransportRequestOptions): Promise<T.IndexDocumentsResponse>;

This is the functions signature. It should index n documents and the limit is 100 if i am not mistaken