How to stop refresh activity while using scroll

Elastic Stack Elasticsearch
1

Team
Can someone help or suggest the way to pause or disable refresh while I update all the docs within scroll request ??

I do see there is a http curl request that we can do against single or multiple indexes but I am looking for a similar to do in Java by using Java api for elastic?? As I am already using it for searching through scroll and updating the docs .

Appreciate a quick reply if possible please

Thanks in advance

2

You can use search_after with point in time (PIT), but it depends on what problem is that you are trying to solve. Can you please elaborate on the problem you are facing?

3

Hi @Christian_Dahlqvist

Thanks for your quick reply

So here is what my utility should be doing
Need to go through many indexes each one by one and also loop through each document and replace an existing field value with a calculated new value. So as there are many docs within an index I am using scroll search request and doing these updates within scroll context and then clearing the scroll once it’s out of all search hits .

So part of this now my question is that if there are any updates coming from other parts of application I should prevent them until I complete this migration job on all docs so want to disable the refresh and then enable it at the end. Part of this I was looking if I can do that via Java api method or something as my complete migration utility was written in Java

Thanks

4

Disabling refresh does not prevent updates from occuring, it is just that any updates will not be searchable.

If you created two different roles, one with read-only access to the index, you could ensure only the user you run the update script access can read and write to the index for a period of time and then switch back to normal access. This would however cause an error for any other process all of a sudden not having write access.

5

Opps sorry its my bad on understanding, I exactly need a config to stop these updates being searchable until my migration is all done on the specific index. Its not that I need a read-only was confused little bit here.

So is there an API call that I can do to disable refresh?

Thanks,

6

What is the problem you are trying to solve (at a higher level)? If you perform updares with refresh interval set to -1 you may overwrite updates that other processes do. Does it matter if you overwrite a document that was indexed during the time it takes for your update to complete (or an update that comes in overwrites an update that you have already performed)?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.