Search timeouts during indexing

Elastic Stack Elasticsearch
12

Thanks, your answered really helped.

The problem was in the frequency of refreshes (of course, looking more deeply, in I/O) and in the fact that django_elasticsearch_dsl does not use the Bulk API by default.

Even '60s' was too much for a large index, and the refreshes took all the 60 seconds or even more, which made the search unresponsive.

I had to:

  • disable all signals in django_elasticsearch_dsl (as thwy were triggering individual actions for every objects;
  • set refresh_interval to '-1' (i.e. to disable automatic refreshes).

Now, I am using bulk updates by specifying the objects that need to be indexed/deleted manually and am refreshing the indices once per week. That resolved the issue.