Temporarily disable ID lookup


I am trying to index an initial workload of ~1 billion documents into ES 5.2. The documents are pulled from an MYSQL database and they already contain a unique ID and I want to use the same IDs in ES for a quick reference.

I was reading about how to tune indexing and I have tried disabling index refresh and replicas and it sped up the process so now I am wondering if there is a way to disable ID lookup as it might speed up things more.

So is there a way to do this?


I don't think so.

But is it really slow? What are you using to do the extraction?

Its not slow per say, however as I move forward with the actual process I will have the shutdown the system to prevent new data from being inserted in the db untill the process finish so I would like to minimize the downtime as much as possible.

There is no need for downtime or prevent new data from being inserted if you use timestamps.

We would like to do the whole tranistion in a single shot without having to worry about new records.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.