# The proper way to reindex a data stream index **URL:** \ **Category:** Elasticsearch\ **Tags:** reindex, datastreams\ **Created:** [November 26, 2024, 8:46pm UTC](https://discuss.elastic.co/t/the-proper-way-to-reindex-a-data-stream-index/371102 "2024-11-26T20:46:17Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![Dave\_Houser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dave_houser/32/72517_2.png) [@Dave\_Houser](https://discuss.elastic.co/u/Dave_Houser)\ **Post date:** [November 26, 2024, 8:46pm UTC](https://discuss.elastic.co/t/the-proper-way-to-reindex-a-data-stream-index/371102/1 "2024-11-26T20:46:17Z")
I have run into an issue with mappings in an data stream index. Can someone help me out? I am getting data type conflicts between the new mapping and old mapping of some fields. What I want to do is just remove the mappings from the older index in the data stream, but apparently that's not possible. It sounds like I have two options, 1. Delete the index (I can't do that it has data we need) 2. Reindex the data stream index. Now I understand reindexing a simple static index, reindexing an index in a DataStream seems to be different process. I tested in my lab reindexing an index, then created an alias for it to the older index, and deleting the older index. Now when I query against the new index or its alias, I find the data. But if I query against the data stream name, I do not find the data. It seems there is no way to "pop" the new index or its alias back into the data stream. I have read online that I could re-index the whole data stream, but that is not practical, there's millions and millions of records, it would take forever. What other options do I have here? ---
**Author:** ![Iron](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/iron/32/139499_2.png) [@Iron](https://discuss.elastic.co/u/Iron)\ **Post date:** [November 27, 2024, 2:00am UTC](https://discuss.elastic.co/t/the-proper-way-to-reindex-a-data-stream-index/371102/2 "2024-11-27T02:00:05Z")
I think you have to create "date view". ---
**Author:** ![Dave\_Houser](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dave_houser/32/72517_2.png) [@Dave\_Houser](https://discuss.elastic.co/u/Dave_Houser)\ **Post date:** [June 25, 2026, 2:03pm UTC](https://discuss.elastic.co/t/the-proper-way-to-reindex-a-data-stream-index/371102/3 "2026-06-25T14:03:56Z")
**UPDATE:** I now have the correct process. This is a real pain to do, and probably should script this. But sharing the current process below. Note: we reindexed to fix wrong data types. The main parts are this - First fix the component template mapping to what you want in elastic. - Rollover the index, and confirm new data is coming in with the correct mappings. - Before reindexing, we needed to create a new `_index_template` taht did not include enrichment pipelines, and set mappings.properties.@timestamp to `ignore_malformed: false` Then include the same settings and mappings. The index pattern should match your new pattern you will use. For us its `.ds-logs-testing-default-*-reindexed-v2` - Then you must create each new index, this will build an empty index with the correct mappings. - Then you can reindex the old index to the new index created with the new mappings - Check count and failure of new indexes - Replace data stream backline with new reindexed reindexes - Delete old indexes when ready. Here are the full details. I hope this help someone else because this is not straight forward the first time you do it. The example includes two indexes that are reindexed, you can add or remove as much as you like. Also, yes you could combine all bad indexes into one big index, but I don't like that, that's why I separate them into separate reindexed indexes. # 1. Prepare Find the index template first "logs-testing-template" ```auto GET _index_template/logs-testing-* ``` Uses logs-testing-settings and logs-testing-mappings ```auto GET _component_template/logs-testing-settings GET _component_template/logs-testing-mappings ``` SAVE - Get a list of all indices in the data stream ```auto GET _data_stream/logs-testing-default?human&filter_path=data_streams.name,data_streams.template,data_streams.generation,data_streams.indices.index_name ``` SAVE - Check mappings of a couple of them before and after the problem ```auto GET .ds-logs-testing-default-2026.06.24-000415/_mapping GET .ds-logs-testing-default-2026.06.04-000373/_mapping ``` Gather mappings of all ```auto GET .ds-logs-testing-default-*/_mapping/field/policy_hit?expand_wildcards=all ``` SAVE - Get count of all documents in the data stream and a couple of indexes ```auto GET logs-testing-default/_count GET .ds-logs-testing-default-2026.06.24-000415/_count GET .ds-logs-testing-default-2026.06.04-000373/_count ``` # 2. Change mappings of current component template ```auto PUT _component_template/logs-testing-mappings { "version": 2, "template": { "mappings": { "properties": { "field01": { "type": "integer" }, "field02": { "type": "integer" }, "field03": { "type": "keyword" }, "field04": { "type": "integer" }, "test": { "type": "keyword" }, "field05": { "type": "integer" }, "field06": { "type": "integer" }, "field07": { "type": "boolean" }, "field08": { "type": "ip" }, "field09": { "type": "keyword" }, "@timestamp": { "type": "date" }, "field10": { "type": "keyword" }, "data_stream": { "properties": { "namespace": { "type": "keyword" }, "type": { "type": "keyword" }, "dataset": { "type": "keyword" } } }, "field11": { "type": "keyword" }, "field12": { "type": "integer" }, "field13": { "type": "ip" }, "field14": { "type": "ip" }, "event": { "properties": { "original": { "type": "text", "index": false } } } } } } } ``` Simulate to confirm it has the correct mappings ```auto POST _index_template/_simulate_index/logs-testing-default?filter_path=template.mappings,template.settings,overlapping ``` Rollover index ```auto POST logs-testing-default/_rollover ``` Check status of data stream ```auto GET _data_stream/logs-testing-default?human&filter_path=data_streams.name,data_streams.generation,data_streams.template,data_streams.indices.index_name ``` # 3. Reindex Reindex all the current mapped indexes Create temp template first. Notice how the enrich pipeline was removed. Also adjust the @timestamp to ignore\_malformed: false ```auto PUT _index_template/logs-testing-reindexed-v2-template { "index_patterns": [".ds-logs-testing-default-*-reindexed-v2"], "template": { "settings": { "index": { "hidden": true } }, "mappings": { "properties": { "@timestamp": { "type": "date", "ignore_malformed": false } } } }, "composed_of": [ "logs-testing-settings", "logs-testing-mappings" ], "priority": 600, "version": 2 } ``` Do a simulate to check to make sure data types apply ```auto POST _index_template/_simulate_index/logs-testing-reindexed-v2-template?filter_path=template.mappings,template.settings,overlapping ``` now create the indexes with the same name matching the new temp index, this will create the correct mappings ```auto PUT /.ds-logs-testing-default-2026.06.05-000375-reindexed-v2 PUT /.ds-logs-testing-default-2026.06.05-000376-reindexed-v2 ``` Check the mappings of a few ```auto GET .ds-logs-testing-default-2026.06.05-000375/_mapping GET .ds-logs-testing-default-2026.06.05-000375-reindexed-v2/_mapping ``` Now reindex all of the old inexes to the new ones created (Run all these POSTS together) ```auto POST _reindex?wait_for_completion=false&slices=auto&requests_per_second=5000 { "source": { "index": ".ds-logs-testing-default-2026.06.05-000375" }, "dest": { "index": ".ds-logs-testing-default-2026.06.05-000375-reindexed-v2", "op_type": "create", "pipeline": "_none" } } POST _reindex?wait_for_completion=false&slices=auto&requests_per_second=5000 { "source": { "index": ".ds-logs-testing-default-2026.06.05-000375" }, "dest": { "index": ".ds-logs-testing-default-2026.06.05-000375-reindexed-v2", "op_type": "create", "pipeline": "_none" } } ``` Check the counts and failures for all of them, if all counts same and no failures, good to swap ```auto GET /.ds-logs-testing-default-2026.06.05-000375-reindexed-v2/_count GET /.ds-logs-testing-default-2026.06.05-000375/_count GET /.ds-logs-testing-default-2026.06.05-000376-reindexed-v2/_count GET /.ds-logs-testing-default-2026.06.05-000376/_count ``` # 4. Cleanup, remove old indexes Remove old backing indexes and replace with new reindexed ones ```auto POST _data_stream/_modify { "actions": [ { "add_backing_index": { "data_stream": "logs-testing-default", "index": ".ds-logs-testing-default-2026.06.05-000375-reindexed-v2" } }, { "remove_backing_index": { "data_stream": "logs-testing-default", "index": ".ds-logs-testing-default-2026.06.05-000375" } }, { "add_backing_index": { "data_stream": "logs-testing-default", "index": ".ds-logs-testing-default-2026.06.05-000376-reindexed-v2" } }, { "remove_backing_index": { "data_stream": "logs-testing-default", "index": ".ds-logs-testing-default-2026.06.05-000376" } } ] } ``` Validate the backing indexes are updated ```auto GET _data_stream/logs-testing-default?human&filter_path=data_streams.name,data_streams.generation,data_streams.indices.index_name ``` WAIT 2 weeks, then Delete old backing indexes with this ```auto DELETE .ds-logs-testing-default-2026.06.05-000375,.ds-logs-testing-default-2026.06.05-000376 ```