Create new field in ES and backfill it with data

(Natalia Kuleniuk) #1

We have ElasticSearch index that stores large dataset. I need to add two new fields to ES index and backfill values for these fields from Redshift. My question is how can I achieve this? What is the best approach: change existed ES scheme by adding 2 new fields or create new index with all needed fields. I am new to ES, so any suggestions, ideas will be helpful.

(Mujtaba Hussain) #2

Lets say you have the two new fields x and y and each document in the current index had the id ID. If you added x and y to the current index, could you match backfilled values to their relevant ids?

(Natalia Kuleniuk) #3

Yes, I thought I can be able to match them. My ElasticSearch has , for example, 2 fields:"A", "B". I would like to add two more fields "C" and "D".
My Redshift table has fields "A", "C" and "D". So I want to copy fields "C" and "D" from Redshift to ES. Both Redshift and ES have "A" field in common.

(David Pilato) #4

You need to reindex. And because you need to reindex all I’d recommend reindexing in a new index.

If you are using aliases which is a good practice, just switch the alias to the new index and drop the old index.

(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.