Documents per shard

Elastic Stack Elasticsearch
1

I've got indexes storing the same kind of data split into weekly chunks -
there has been some fairly substantial variation in data volume.
I've got a mapping change I need to make across all the back data, and I'm
thinking it might make sense to try to rebalance the documents per shard so
that I have around 1 shard per N documents.
Is that a worthwhile time investment in terms of query performance, or
should I just stick with the 3 shards per index I've been using so far? I'd
keep 3 shards as a minimum, so if there's a week with 10 documents it would
still have 3 shards.

If I have an index that would end up with more than one shard per data
node, does it make more sense to limit the number of shards to the number
of data nodes, or go ahead and follow the 1 shard per N documents pattern?

Thanks!
Josh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d775acd2-9717-4fe6-bbda-9c6d42f0cb39%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

2

You're probably the best person to answer that simply because you know your
data. But it sounds like if you know that you won't exceed N documents per
index (or averaging N docs per index), then you just have to find the
optimal number of shards that will produce the right performance for your N
documents. And that may well be 1 shard only. I do think that it would be a
waste of resources to have an index of 3 shards and only have 10 documents
(or even a few hundred documents) in that index.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9ec6f2dc-3de2-4b47-9618-bdf4bd0b6b51%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.