Migrate to datastreams withou aliases + datastreams filter

Elastic Stack Elasticsearch
1

Hello,
I'd like to use data streams and migrate our indices to it. However currently we don't use aliases. Do I have to create an alias for given indices pattern (technical-logs*) at first and only after that I'm able to migrate to data streams?

Second question is more about filtering. When we are filtering by date we just get all indices names from ES and then we search in those indices which match the given filter (e.g. from technical-logs-20230101 to technical-logs-20230118). When we switch to data streams it seems like the indices will be hidden. So the question is am I able to somehow specify in which indices should query search? Or is it all automatically being applied when I specify @timestamp parameter?

Thanks

2

Welcome to our community! :smiley:

Generally you would reindex older data into the data stream so you don't need to worry about this.

It might be easier to provide a bit more context on how you are querying, because generally that's not what you'd do and is inefficient.

3

the question was more regarding this topic. Documentation tells only about migration from index alias. So if I have many indices (technical-logs-20230101, technical-logs-20230102, ...) which I want to migrate to one data stream am I able to do that without creating an alias for those indices? Currently there is no alias.

So we have many groups of indices (technical-logs*, notifications*, incidents*, ...). We do a rollover daily so there is always date suffix. If we don't provide the date range, we are searching in indices with asterisk (e.g. technical-logs*). If we provide date range then we are trying to get all indices which are in given range (e.g. from technical-logs-20230101 to technical-logs-20230118 ). Now the question is if this is really necessary and if ES doesn't apply such filter automatically when we filter by @timestamp.

4

Ah ok. I don't know to be honest. If you follow that through to Migrate to data stream API | Elasticsearch Guide [7.17] | Elastic it says;

If successful, the request removes the alias and creates a data stream with the same name. The alias’s indices become hidden backing indices for the stream. The alias’s write index becomes the stream’s write index.

So my guess it doesn't do much other than some alias switching/renaming.

Elasticsearch handles this transparently for you, so I wouldn't worry too much.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.