Hi,
I have an index that our application is working with like a Relational table,
so we insert and update documents in it.
now we want to make a structure change and add 3 fields and add data to those fields from a table in SQL Server, based on ID that we have in Elasticsearch like so:
A_ID (ES) -> A_ID (MSSQL) -> take A_Name from MSSQL and update A_Name in ES.
(something like this logic:
UPDATE es.index SET A_Name = mssql.A_Name WHERE A_ID = mssql.A_ID)
the same goes for other fields, we have the ID in ES and MSSQL and we want to update the Name in ES based on the value in MSSQL.
we thought about doing it using update_by_query but because it's a heavy process we though about doing it using _reindex...
the issue is that _reindex is INSERTing or REPLACEing documents,
the question is if it's possible to build a pipeline that would make the _reindex update those 3 fields without losing all other fields in each document?
(assuming we would bring the ID and the Name from MSSQL to ES as an index)
We cannot use enrichment...