Adding new field to an existing index/dataview

Elastic Stack Kibana
1

version 8.18.3

I found a field in one datastream where the name in the index template did not match the one that was used in the ingested data. I have added the ‘used’ name into the index template and rotated the datastream.

I still can not add the field to the dataview because there are now type inconstancies for that field.

Should I now POST _migration/reindex …. to fix the type issues?

Russell

2

I don’t think that POST _migration/reindex will help you – it is meant for reindexing a data stream for upgrade, and will just reindex each index with the same mappings it already has.

I’m not sure what the dataview looks for in order to decide if it will let you add a field (maybe a kibana expert will see this). How many indices do you have with the old mapping for the field? Is it feasible to reindex each of those old indices with the new correct mapping?

3

Is this showing up as a field conflict in Kibana when you look at the Data View in Stack Management?

If you have a field conflict you have basically 2 options, one is to remove the indices where the field has conflicting data types, the other one is to reindex all indices that have conflicting data type using the Reindex API.

4

thanks Keith. I was puzzled that is why I posted – it was an “AI” suggestion (Google) and it looked to good to be true.

the datastream has about 20 indexes – I assume I would have to set up a new datastream and reindex into that then delete the old one. Doable but changing all the dashboards etc would be very painful.

5

You don’t have to set up a new data stream. Just create a new index for each of the bad ones, make sure it has the correct settings and mappings, reindex the bad ones, and then swap the new ones into the data stream and the old ones out. It won’t be pleasant, but you won’t have to change your data stream name or your dashboards.