How does the elasticsearch expand capacity and shrink capacity?

(Alice) #1

elasticsearch version: 5.5.2
OS: Linux
A cluster of 3 nodes.

How does elasticsearch reduce capacity and scale out capacity? Is there any operation command?

(Christian Dahlqvist) #2

What do you mean by expanding and shrinking capacity?

(Alice) #3

Sorry, I mean how elasticsearch reduce capacity and scale out capacity.

(Christian Dahlqvist) #4

What type of capacity? Storage, nodes, heap, RAM, CPU?

(Alice) #5

Yes, it is storage. Now I put large amount of data in the index, but the primary shards can't store any more, how to solve this problem?

(Christian Dahlqvist) #6

What is your use case? How many indices and shards do you have? How many documents in total? How large are the indices and shards?

(Alice) #7

Yeah, thanks. I put 10TB data into one index, and there are 3 primary shards and 6 replica shards, if I want to store more data, how to scale out the storage of elasticsearch.

(Christian Dahlqvist) #8

What type of data do you have? What is the use case? I assume the PB is a typo? Should it be GB or TB?

(Alice) #9

Yeah, the unit is PB. Because the number of primary shards can't be modified once created, then how do we scale out their storage later?

(Christian Dahlqvist) #10

It looks like you corrected it to TB. What is the use case? What type of data do you have? The reason I am asking this is that it impacts how you best manage your data.

Setting number_of_shards but couldn't resolve host
(Christian Dahlqvist) #11

I would recommend you read this section from 'Elasticsearch: the definitive guide' around designing for scale.

(Alice) #12

Sorry, I don't have particular type of data now, I just assumpt that.

(Alice) #13

Thanks, but there aren't many operations guides in the page. I'm sorry that I still have no idea how to do.

(Christian Dahlqvist) #14

You define your indexing and sharing strategy based on what type of data you have, which is described in the documentation I linked to. Elasticsearch will use the storage it has available and once you reach the limit of the nodes in the cluster, you usually scale up/and or out, at which point Elasticsearch will redistribute the data.

Once you have more details we might be able to provide better and more detailed guidance.

(Alice) #15

Thanks, we really looking forward to your more detailed guidance. And could you explain a lit more detail about how Elasticsearch redistribute the data?

(Christian Dahlqvist) #16

Please read the full chapter I linked earlier. You may also find this guide useful.

Without details around your use-case or data I am afraid I can not provide any more detailed guidance.

(Alice) #17

Ok, thanks a lot.

(system) #18

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.