How to import 10 millions of documents (about 5G) into app search?

@werewolf_ninja FWIW, you should be getting much better throughput than 5000 documents every 30 minutes. From what I can see, we should be allowing up to 3000 documents per minute, assuming you are using a paid App Search account.

You can use the following headers in the response for reference:

X-RateLimit-Limit → <number of documents allowed in 1 minute>
X-RateLimit-Remaining → <number of documents remaining this minute>

If you are rate limited and receive a 429 response, you can check the Retry-After header which will tell you how long to wait before performing a new request.

1 Like