Missing Mapping caused by an updated Translation file - Possible to Reindex without Restart all?

(Thorben) #1

Hi there,

I just checked the ES Put Mapping API but that's like "Böhmische Dörfer" for me atm.
What I try to do is following:

My ES Index name = "dict_c"

In my logstash conf:

translate {
    		field => "[Präfix_Country]"
    		destination => "[Country_New]"
    		override => true
    		dictionary_path => "C:/dict/dict_countrys.csv"
    		}

Now, the translation file will be updated sometime.
Let's say:

from "+049","DE"
to "+049","GERMANY"

And thats my missing link :slight_smile:
How to reindex or re-mapping [Country_New] on this specific task wihout restart all.
The Documentation is not giving me the right hint, honestly, I didn't understand it that well at this point either.

Thanks for any help and Regards

(Guy Boertje) #2

Do you mean, go back into the ES documents indexed with the old value and change it to the new value?

(Thorben) #3

Hi,
yes this is what I want to achive!
No structure adjustments or changes, just the values of extisting fields of an existing index.

Thanks

(Guy Boertje) #4

In another pipeline, you will need to use the Elasticsearch Input to query for the docs that have the old value, push them through the translate filter again and do an update on the Elasticsearch output.

By what mechanism is the translate dictionary updated?

(Thorben) #5

Hi,
ok I'll do as per your instructions.
There is a master csv and the files will be simply replaced of it.

(Guy Boertje) #6

I mean is the master CSV file updated by a human or a script? What triggers the update?
Do you record the date/time the update is done - you only need to re-index docs ingested before that date/time.
What will you do about really old records that have a looked up value from the master CSV file three (more) updates ago?
You only need to run the reindex LS pipeline once after each update to the master CSV file, how will you do this? Via config reloading, Centralised Pipeline Management or manually starting up a LS instance?

(Thorben) #7

Hi,
sorry for absence!
The files will be updated manually.
Fully undertstand your advice.
Its in (learning) progress, will come back if I have more questions.
Regards