Need a hep on how to choose a particular shard to index a document to. I know about the routing concept that we can let similar type of documents to reside in same shard. But this is what happening with our production environment of elasticsearch:
A 3 data cluster node which has 32 primary shards. The data is basically time series data and we try to group together data to a month in a single shard using the routing principle. This works absolutely fine but what we are observing is that certain shards contains most of months whereas certain shards just contain a single month or so.
The result is if we want to work on certain months of data, then we cannot utilize the power of sharding since the query goes to a single node only.
Is there a way where in along with routing value I can also specify this is the shard I want my data to reside in or at least some functionality where ES distributes data according to our need.