If i have large amount of data, should i create multi data stream instead of a single data stream

Elastic Stack Elasticsearch
#1

If the storage capacity is about several Trillionbyte, should i use a single data stream?
like this:

data stream aaa

contains index:aaa-2022.01.06-0001,aaa-2022.01.06-0002,aaa-2022.01.07-0003

or several data stream

data stream one: aaa-2022.01.06,constains index:aaa-2022.01.06-2022.01.06-0001

data stream two: aaa-2022.01.07,constains index:aaa-2022.01.07-2022.01.07-0001

#2

Just the one is fine. Datastreams use ILM under the hood, so they will roll indices as needed.

#3

Multiple shards can be used to scale ingest also. IMHO, you should not have more shards than nodes, never force multiple actively writing shards on a single node.

#4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.