Impact of multiple concurrent scroll API on ElasticSearch

Elastic Stack Elasticsearch
1

I am using a large number of search API(using scroll = '1m') to search documents from it

Average no of requests per minute = 500
Average time of hit = 100 ms

Sometimes when hits are at peak I started getting error

{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred"}

Is it because of scroll timing ? or any other problem.
Please Suggest !!

2

Scrolls are expensive to run concurrently.
For each scroll ID there is a unique point-in-time view of the current set of segments preserved for that scroll. This hangs on to files and related caches that would otherwise be removed by the constant segment rewriting that happens while indexing is active. This is why it is especially resource-intensive to do concurrently.

1 Like
3

Yeah Scroll reserves the resource for that particular time, that was the problem I solved it already.
Anyways Thanks..

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.