So I have a requirement to index some data repeatedly. Each time the data might have different data fields. I want to keep the superset of the fields in the index. I.e.
initial load the document like
{"_id": 100, "field1": "value1", "field2": "value2"}
so it will be created.
2nd time I might load as
{"_id": 100, "field2": "value2", "field3": "value3"}
looks like the document got updated and "field1" got removed. Is there an append way to bulk loading?
The tricky part is I would not now if each _id exists or not. We required to use a specific column as ID. I don't want to row by row to see if the _id exists or not as we have a lot of data.
Thank you.
-D