Unexpected shard allocation with 0 replica indexes

Hi All,
We have several indexes in our ES cluster. ES is not our canonical
system record, we use it only for searching.

Some of our applications have very high write throughput, so for these we
allocate a singular primary shard for each of our nodes. For example, we
have 6 nodes, and we create our index with 6 shards and 0 replicas. I
would expect ES to balance primary shards per node in the index. However,
it will load as many as 3 primaries on a single node, and some nodes will
have no shards. While each node does appear to have the same number of
primaries, not all our indexes have equal write throughput, therefore, I
want the primary (and only) shards evenly distributed per index.

During heavy writes, we experience slowness because our shards are not
evenly distributed across our nodes. Currently we're getting around this
by manually moving shards, but this seems to be something ES should do
automatically. We're running version 1.4.3 with default allocation
parameter. Is this a bug?

Thanks,
Todd

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5cc33cd3-f7f6-4174-b815-cadc858c0b97%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

It's not a bug, ES allocates based on higher level primary shard count and
doesn't take into account what index a shard may belong to, this is where
allocation awareness and routing comes into play.

Take a look at
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-awareness

On 23 February 2015 at 06:46, Todd Nine tnine@apigee.com wrote:

Hi All,
We have several indexes in our ES cluster. ES is not our canonical
system record, we use it only for searching.

Some of our applications have very high write throughput, so for these we
allocate a singular primary shard for each of our nodes. For example, we
have 6 nodes, and we create our index with 6 shards and 0 replicas. I
would expect ES to balance primary shards per node in the index. However,
it will load as many as 3 primaries on a single node, and some nodes will
have no shards. While each node does appear to have the same number of
primaries, not all our indexes have equal write throughput, therefore, I
want the primary (and only) shards evenly distributed per index.

During heavy writes, we experience slowness because our shards are not
evenly distributed across our nodes. Currently we're getting around this
by manually moving shards, but this seems to be something ES should do
automatically. We're running version 1.4.3 with default allocation
parameter. Is this a bug?

Thanks,
Todd

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5cc33cd3-f7f6-4174-b815-cadc858c0b97%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5cc33cd3-f7f6-4174-b815-cadc858c0b97%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8Ue_1SYTud9HZdC5aGHNOUJVA%2Bq8PT8LrQrzm7GaOZ7Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Thanks Mark,
I've looked at these, as well as these configuration parameters.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-update-settings.html#_balanced_shards

However, I've not been able to find a configuration that actually
accomplishes what we're looking for. We're using the aws plugin as well,
which doesn't seem to help any with our shard allocation. It does however,
ensure we have replica's in other zones, which is obviously helpful for
redundancy.

Are there any known settings, either with allocation-awareness or cluster
settings that will accomplish the behavior I'm looking for? Otherwise,
we'll have to write client API to move the shards after index allocation
to ensure we get our even load.

Thanks,
Todd

On Sunday, February 22, 2015 at 3:25:28 PM UTC-7, Mark Walkom wrote:

It's not a bug, ES allocates based on higher level primary shard count and
doesn't take into account what index a shard may belong to, this is where
allocation awareness and routing comes into play.

Take a look at
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-awareness

On 23 February 2015 at 06:46, Todd Nine <tn...@apigee.com <javascript:>>
wrote:

Hi All,
We have several indexes in our ES cluster. ES is not our canonical
system record, we use it only for searching.

Some of our applications have very high write throughput, so for these we
allocate a singular primary shard for each of our nodes. For example, we
have 6 nodes, and we create our index with 6 shards and 0 replicas. I
would expect ES to balance primary shards per node in the index. However,
it will load as many as 3 primaries on a single node, and some nodes will
have no shards. While each node does appear to have the same number of
primaries, not all our indexes have equal write throughput, therefore, I
want the primary (and only) shards evenly distributed per index.

During heavy writes, we experience slowness because our shards are not
evenly distributed across our nodes. Currently we're getting around this
by manually moving shards, but this seems to be something ES should do
automatically. We're running version 1.4.3 with default allocation
parameter. Is this a bug?

Thanks,
Todd

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5cc33cd3-f7f6-4174-b815-cadc858c0b97%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5cc33cd3-f7f6-4174-b815-cadc858c0b97%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8a6b76c7-87f0-4a2f-a08f-db9a6fb871f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.