Removing one shard of a pair


(Kenneth Loafman) #1

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per index. Is
there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Shay Banon) #2

auto_expand_replcias is really only meaningful when you have a single shard
index.

Back to your case, with 2 shards and 1 replica, you have a total of 4
shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman
kenneth.loafman@gmail.comwrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Kenneth Loafman-2) #3

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.com wrote:

auto_expand_replcias is really only meaningful when you have a single
shard index.

Back to your case, with 2 shards and 1 replica, you have a total of 4
shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Shay Banon) #4

Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.com wrote:

auto_expand_replcias is really only meaningful when you have a single
shard index.

Back to your case, with 2 shards and 1 replica, you have a total of 4
shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Kenneth Loafman-2) #5

OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:

Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.com wrote:

auto_expand_replcias is really only meaningful when you have a single
shard index.

Back to your case, with 2 shards and 1 replica, you have a total of 4
shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Shay Banon) #6

But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...

On Tue, Jan 24, 2012 at 5:30 PM, Kenneth Loafman kenneth@loafman.comwrote:

OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:

Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.com wrote:

auto_expand_replcias is really only meaningful when you have a single
shard index.

Back to your case, with 2 shards and 1 replica, you have a total of 4
shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Kenneth Loafman-2) #7

Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?

On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:

But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...

On Tue, Jan 24, 2012 at 5:30 PM, Kenneth Loafman kenneth@loafman.comwrote:

OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:

Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.com wrote:

auto_expand_replcias is really only meaningful when you have a single
shard index.

Back to your case, with 2 shards and 1 replica, you have a total of 4
shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Shay Banon) #8

Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local:
http://www.elasticsearch.org/guide/reference/api/search/preference.html.

On Tue, Jan 24, 2012 at 6:05 PM, Kenneth Loafman kenneth@loafman.comwrote:

Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?

On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:

But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...

On Tue, Jan 24, 2012 at 5:30 PM, Kenneth Loafman kenneth@loafman.comwrote:

OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:

Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.com wrote:

auto_expand_replcias is really only meaningful when you have a single
shard index.

Back to your case, with 2 shards and 1 replica, you have a total of 4
shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Kenneth Loafman-2) #9

Thanks. One more question, what is the shard size limit?

On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:

Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local:
http://www.elasticsearch.org/guide/reference/api/search/preference.html.

On Tue, Jan 24, 2012 at 6:05 PM, Kenneth Loafman kenneth@loafman.comwrote:

Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?

On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:

But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...

On Tue, Jan 24, 2012 at 5:30 PM, Kenneth Loafman kenneth@loafman.comwrote:

OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:

Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman kenneth@loafman.comwrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.com wrote:

auto_expand_replcias is really only meaningful when you have a
single shard index.

Back to your case, with 2 shards and 1 replica, you have a total of
4 shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Shay Banon) #10

There isn't really a limit to a shard size, but, it might exhaust itself in
several aspects such as search perf, amount of memory used. It really
depends on what you index and how you search to guess what a good shard
size is.

On Tue, Jan 24, 2012 at 7:14 PM, Kenneth Loafman kenneth@loafman.comwrote:

Thanks. One more question, what is the shard size limit?

On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:

Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local:
http://www.elasticsearch.org/guide/reference/api/search/preference.html.

On Tue, Jan 24, 2012 at 6:05 PM, Kenneth Loafman kenneth@loafman.comwrote:

Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?

On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:

But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...

On Tue, Jan 24, 2012 at 5:30 PM, Kenneth Loafman kenneth@loafman.comwrote:

OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:

Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman <kenneth@loafman.com

wrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.comwrote:

auto_expand_replcias is really only meaningful when you have a
single shard index.

Back to your case, with 2 shards and 1 replica, you have a total of
4 shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Kenneth Loafman-2) #11

Thanks.

So it's a bit of magic. I was afraid of that.

On Tue, Jan 24, 2012 at 11:34 AM, Shay Banon kimchy@gmail.com wrote:

There isn't really a limit to a shard size, but, it might exhaust itself
in several aspects such as search perf, amount of memory used. It really
depends on what you index and how you search to guess what a good shard
size is.

On Tue, Jan 24, 2012 at 7:14 PM, Kenneth Loafman kenneth@loafman.comwrote:

Thanks. One more question, what is the shard size limit?

On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:

Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local:
http://www.elasticsearch.org/guide/reference/api/search/preference.html.

On Tue, Jan 24, 2012 at 6:05 PM, Kenneth Loafman kenneth@loafman.comwrote:

Maybe an understanding... From the discussion on the board, I got that
a replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?

On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:

But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...

On Tue, Jan 24, 2012 at 5:30 PM, Kenneth Loafman kenneth@loafman.comwrote:

OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:

Yes, though you can't remove the setting, you can set the auto
expand setting to false (assuming you work on a live cluster and use the
index update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman <
kenneth@loafman.com> wrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.comwrote:

auto_expand_replcias is really only meaningful when you have a
single shard index.

Back to your case, with 2 shards and 1 replica, you have a total
of 4 shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(Kenneth Loafman-2) #12

Back to preference, is there an example somewhere on how to use it? I need
to set it to _local since pyes does a round-robin load balancing.

On Tue, Jan 24, 2012 at 12:43 PM, Kenneth Loafman kenneth@loafman.comwrote:

Thanks.

So it's a bit of magic. I was afraid of that.

On Tue, Jan 24, 2012 at 11:34 AM, Shay Banon kimchy@gmail.com wrote:

There isn't really a limit to a shard size, but, it might exhaust itself
in several aspects such as search perf, amount of memory used. It really
depends on what you index and how you search to guess what a good shard
size is.

On Tue, Jan 24, 2012 at 7:14 PM, Kenneth Loafman kenneth@loafman.comwrote:

Thanks. One more question, what is the shard size limit?

On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:

Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local:
http://www.elasticsearch.org/guide/reference/api/search/preference.html
.

On Tue, Jan 24, 2012 at 6:05 PM, Kenneth Loafman kenneth@loafman.comwrote:

Maybe an understanding... From the discussion on the board, I got that
a replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?

On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:

But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...

On Tue, Jan 24, 2012 at 5:30 PM, Kenneth Loafman <kenneth@loafman.com

wrote:

OK, thanks for the help. I think having one shard and
auto-expanding replicas is the way to go since we'll be increasing the
cluster as we go forward.

On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.comwrote:

Yes, though you can't remove the setting, you can set the auto
expand setting to false (assuming you work on a live cluster and use the
index update settings API).

On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman <
kenneth@loafman.com> wrote:

So if I remove the auto_expand_replicas, it should go back to 4?

On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.comwrote:

auto_expand_replcias is really only meaningful when you have a
single shard index.

Back to your case, with 2 shards and 1 replica, you have a total
of 4 shards, so on a 4 node cluster, you will have 1 shard per node.

On Tue, Jan 24, 2012 at 4:35 PM, Kenneth Loafman <
kenneth.loafman@gmail.com> wrote:

Hi,

Originally we had the following on a 3-node cluster:

index :
number_of_shards : 2
number_of_replicas : 1

Then we increased the cluster to 4 nodes and modded the setup to:

index :
number_of_shards : 2
number_of_replicas : 1
auto_expand_replicas : "0-all"

The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?

...Thanks,
...Ken


(system) #13