ElasticSearch cannot perform large searches when the index is being updated

My server has a MongoDB database, ElasticSearch, Monstache (keeps the MongoDB database and ElasticSearch index in sync), and a Node.js REST API running at the same time. Users are conducting searches via a web app. The web app sends the search query to the API, which sends request(s) to ElasticSearch to get the search results. Large searches use ElasticSearch's search_after feature to retrieve the results in batches (pagination).

When Monstache is updating the ElasticSearch index, ElasticSearch fails to return any more batches of search results past a certain point. So for example, the API will retrieve 80-90% of the results in batches; then when it tries to retrieve the next batch, only part of the batch is returned; then when it tries to retrieve the next batch, it receives an empty batch and the API crashes.

Is ElasticSearch not designed to search and update an index at the same time? Why can't I successfully search with ElasticSearch while the index is being updated?

search_after doesn't guarantee that it will always work on a consistent snapshot of the data, so if you change the data while it's running then you may retrieve fewer results than you expected. But "crash" is a strong word. What do you mean by "the API crashes"? Can you share the error you're seeing?

Hi David,

Thanks so much for your reply, and apologies for my delayed response. I tried monitoring Elasticsearch's index size while the API conducted a large (paginated) search, and the index size was not changing... so Monstache was updating documents during the search, not deleting them. Does this mean that, if a document is updated (not deleted) during search_after, it will be dropped from the search results?

I'm sorry but I'm unable to recreate the API error at this time. I don't think it matters though... I've already edited the API code to stop it from crashing when this happens, so instead it is just returning whatever results it can get. I am more concerned with returning accurate search results at this point. :slight_smile:

This reasoning doesn't quite work. Updates are implemented internally using deletes, and both updates and deletes should increase the size of an index at least until a merge happens. You would be better off using the indices stats API for this.

No, that's unexpected. Can you give an example of the search_after query you're running?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.