Hi,
We are using elasticsearch as our primary datastore and we are constantly updating the elasticsearch indexes, using worker processes.
Before inserting, we have to check if another record with the same field is already in the database, otherwise we dont write the duplicate. For this purpose, after each write to the elasticsearch, we are doing an index refresh so that we will get the original while searching the database (if index is not refreshed, the record will not come in search, if it is encountered within 1 second, which is the refresh rate of elasticsearch). But this is affecting the insert rate of the database. Is there any better way to check the uniqueness of a field in the database?