Need help with shard balancing

We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the other
one is empty. One of the nodes it doing very little work because it has
mostly empty shards.

I need a way to have ES distribute the shards from each index evenly across
all 4 nodes. I know by default, ES considers all shards equally. I've
looked into some of the ES config settings around shard balancing, but it's
not obvious how to make this happen.

Any pointers appreciated, thanks!

-Craig

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Craig

On Wed, 2013-02-06 at 08:26 -0800, Craig Brown wrote:

We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the
other one is empty. One of the nodes it doing very little work because
it has mostly empty shards.

I need a way to have ES distribute the shards from each index evenly
across all 4 nodes. I know by default, ES considers all shards
equally. I've looked into some of the ES config settings around shard
balancing, but it's not obvious how to make this happen.

If you're using v0.19.10 or higher, then you can use the cluster reroute
API to move your shards around:

http://www.elasticsearch.org/guide/reference/api/admin-cluster-reroute.html

If not, and you're not using the other index, try just closing and
reopening it. If that doesn't work, you may have to delete it

clint

Any pointers appreciated, thanks!

-Craig

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

The best way to do this is via
index.routing.allocation.total_shards_per_node:
http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.html

I have a process that looks at the number of data nodes I have and the
number of shards for an index and automatically sets this to force a fairer
balancing. Would be a nice feature to have in the core elasticsearch,
though.

Best Regards,
Paul

On Wednesday, February 6, 2013 9:45:02 AM UTC-7, Clinton Gormley wrote:

Hi Craig

On Wed, 2013-02-06 at 08:26 -0800, Craig Brown wrote:

We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the
other one is empty. One of the nodes it doing very little work because
it has mostly empty shards.

I need a way to have ES distribute the shards from each index evenly
across all 4 nodes. I know by default, ES considers all shards
equally. I've looked into some of the ES config settings around shard
balancing, but it's not obvious how to make this happen.

If you're using v0.19.10 or higher, then you can use the cluster reroute
API to move your shards around:

http://www.elasticsearch.org/guide/reference/api/admin-cluster-reroute.html

If not, and you're not using the other index, try just closing and
reopening it. If that doesn't work, you may have to delete it

clint

Any pointers appreciated, thanks!

-Craig

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Thanks. Yes, its a 19.9 cluster.

  • Craig

On Wed, Feb 6, 2013 at 10:03 AM, ppearcy ppearcy@gmail.com wrote:

The best way to do this is via
index.routing.allocation.total_shards_per_node:

http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings.html

I have a process that looks at the number of data nodes I have and the
number of shards for an index and automatically sets this to force a fairer
balancing. Would be a nice feature to have in the core elasticsearch,
though.

Best Regards,
Paul

On Wednesday, February 6, 2013 9:45:02 AM UTC-7, Clinton Gormley wrote:

Hi Craig

On Wed, 2013-02-06 at 08:26 -0800, Craig Brown wrote:

We have a 4-node cluster with 2 indexes each with 1 replica. We are
currently indexing all documents against a single index, while the
other one is empty. One of the nodes it doing very little work because
it has mostly empty shards.

I need a way to have ES distribute the shards from each index evenly
across all 4 nodes. I know by default, ES considers all shards
equally. I've looked into some of the ES config settings around shard
balancing, but it's not obvious how to make this happen.

If you're using v0.19.10 or higher, then you can use the cluster reroute
API to move your shards around:

http://www.elasticsearch.org/guide/reference/api/admin-
cluster-reroute.htmlhttp://www.elasticsearch.org/guide/reference/api/admin-cluster-reroute.html

If not, and you're not using the other index, try just closing and
reopening it. If that doesn't work, you may have to delete it

clint

Any pointers appreciated, thanks!

-Craig

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@**googlegroups.com.
For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

CRAIG BROWN
Search Architect
youwho, Inc.

www.youwho.com

M: 801.680.7300

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.