Is there any way to get elasticsearch to automatically refresh an index even if it never gets any queries? The reason I need to do this is I have a process which is constantly running, updating my index with delta updates (updateByQuery etc)...in the situation that I need to migrate to a new index I create a secondary index, then I continuously issue all my write operations to both indexes, but always read from the primary index...I do a sync process which repopulates all the data into both indexes, then when complete I switch the secondary (new) index to be the primary, and delete the original index. The problem is, when I'm in the mode where it's issuing the writes to both indexes, the secondary one gets out of date and starts spitting conflicts, and since queries are only going to the primary index the secondary one never refreshes automatically. I know I can explicitly tell it to refresh periodically...or I can set a timer that just sends dummy queries to the secondary index every 20 secs or so, but I'm hoping there's a more elegant configuration setting or something I can leverage to accomplish this. Thx.
What version are you on?
https://www.elastic.co/guide/en/elasticsearch/reference/7.10/near-real-time.html is how they work in 7.10 for eg.
I am currently running on 7.1, but 7.10 works the same way....from the link you shared:
By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds. This is why we say that Elasticsearch has near real-time search: document changes are not visible to search immediately, but will become visible within this timeframe.
So, unless the index receives a "search request", it will never refresh...and unfortunately it seems like updateByQuery and deleteByQuery do not count as "search requests" for this purpose.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.