Sharding in ES


(Vaibhav Gulati) #1

"Each document in your
index belongs to a single primary shard, so the number of primary shards that you
have determines the maximum amount of data that your index can hold."

can anyone help me understand this?


(David Pilato) #2

If a single shard can hold for example for your use case 1m documents and you have 5 primary shards, you'll be able to index up to 5m documents in your index before it reaches the limits you have.


(Vaibhav Gulati) #3

What if my harddisk is of 5GB, still I would be able to store 5m of data in my index?


(David Pilato) #4

It depends. But for a use case I have, I'm indexing around 250 mb for 1m documents.


(Christian Dahlqvist) #5

I think Lucene limits the number of documents per shard to around 2 billion, so shards can get quite big. You are however likely to find that your ideal shard size based on query performance is smaller than that.


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.