Datastream : reindex an indice

Elastic Stack Elasticsearch
1

Hi,

I have a datastream named "logs-local1-default" that have 3 indices:

  • .ds-logs-local1-default-2024.12.11-000001
  • .ds-logs-local1-default-2025.02.15-000002
  • .ds-logs-local1-default-2025.03.25-000003

One (the second one) need to be reindexed because 2 fields were not detected correctly (string vs integer).

I already tried to reindex the indice using logstash with this script:

input {
    elasticsearch {
        hosts => "https://elk.myserver.fr:9200"
        index => ".ds-logs-local1-default-2025.02.15-000002"
        ssl_enabled => true
        ssl_truststore_path => "/etc/logstash/certs/elastic-stack-ca.p12"
        ssl_truststore_password => "myPassword"
        ssl_keystore_path => "/etc/logstash/certs/http.p12"
        ssl_keystore_password => "myPassword"
        user => "logstash_writer"
        password => "logstash"
        size => 1000
        scroll => "5m"
        docinfo => true
    }
}

output {
    elasticsearch {
        hosts => "https://elk.myserver.fr:9200"
        index => ".ds-logs-local1-default-2025.02.14-000002"
        ssl_enabled => true
        ssl_truststore_path => "/etc/logstash/certs/elastic-stack-ca.p12"
        ssl_truststore_password => "myPassword"
        ssl_keystore_path => "/etc/logstash/certs/http.p12"
        ssl_keystore_password => "myPassword"
        user => "logstash_writer"
        password => "logstash"
    }

    stdout {
        codec => "dots"
    }
}

The new indice was created with the correct fields but I don't know how to "add" it to the datastream.

Or is there another way to do it ?
What's the best way ?

Any advice is welcome.

Thanks

2

Is

What you are looking for?

3

Perfect !

Thanks a lot.

4

If you are using ILM, you'll also want to copy over the index.lifecycle.name setting. Unfortunately though, the ILM state for the index is kept in the cluster state (an internal elasticsearch data structure), which makes it pretty difficult to copy over to the new index.
It doesn't help you in your current situation, but we have recently added an API to reindex old indices within a data stream to assist with upgrades. There are more edge cases than you would expect. It's possible that that API could be expanded in the future to cover he more general use case you have. But there are no concrete plans for that right now.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.