The problem is that this increases the number of shards to 8 per index. Is
there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?
The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?
The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?
Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).
The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?
OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:
Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).
The problem is that this increases the number of shards to 8 per index.
Is there a way to set number_of_shards to 1, either automatically, or
manually? Am I going about this the wrong way if I want a shard on each
node?
But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...
OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:
Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).
The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?
Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?
On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:
But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...
OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:
Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).
The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?
Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local: Elasticsearch Platform — Find real-time answers at scale | Elastic.
Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?
On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:
But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...
OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:
Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).
The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?
Thanks. One more question, what is the shard size limit?
On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:
Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local: Elasticsearch Platform — Find real-time answers at scale | Elastic.
Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?
On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:
But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...
OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:
Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).
The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?
There isn't really a limit to a shard size, but, it might exhaust itself in
several aspects such as search perf, amount of memory used. It really
depends on what you index and how you search to guess what a good shard
size is.
Thanks. One more question, what is the shard size limit?
On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:
Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local: Elasticsearch Platform — Find real-time answers at scale | Elastic.
Maybe an understanding... From the discussion on the board, I got that a
replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?
On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:
But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...
OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:
Yes, though you can't remove the setting, you can set the auto expand
setting to false (assuming you work on a live cluster and use the index
update settings API).
The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?
On Tue, Jan 24, 2012 at 11:34 AM, Shay Banon kimchy@gmail.com wrote:
There isn't really a limit to a shard size, but, it might exhaust itself
in several aspects such as search perf, amount of memory used. It really
depends on what you index and how you search to guess what a good shard
size is.
Thanks. One more question, what is the shard size limit?
On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:
Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local: Elasticsearch Platform — Find real-time answers at scale | Elastic.
Maybe an understanding... From the discussion on the board, I got that
a replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?
On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:
But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...
OK, thanks for the help. I think having one shard and auto-expanding
replicas is the way to go since we'll be increasing the cluster as we go
forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.com wrote:
Yes, though you can't remove the setting, you can set the auto
expand setting to false (assuming you work on a live cluster and use the
index update settings API).
On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman < kenneth@loafman.com> wrote:
So if I remove the auto_expand_replicas, it should go back to 4?
On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.comwrote:
auto_expand_replcias is really only meaningful when you have a
single shard index.
Back to your case, with 2 shards and 1 replica, you have a total
of 4 shards, so on a 4 node cluster, you will have 1 shard per node.
The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?
On Tue, Jan 24, 2012 at 11:34 AM, Shay Banon kimchy@gmail.com wrote:
There isn't really a limit to a shard size, but, it might exhaust itself
in several aspects such as search perf, amount of memory used. It really
depends on what you index and how you search to guess what a good shard
size is.
Thanks. One more question, what is the shard size limit?
On Tue, Jan 24, 2012 at 11:03 AM, Shay Banon kimchy@gmail.com wrote:
Yes, you do a single shard and have the replicas spread across the
remaining nodes. But then your index is bounded in size to what a single
shard can hold. Btw, by default, when you hit a node in the cluster, even
if a shard exists on it, it might go and execute the request on another
node (because it can't tell if you are doing loadbalancing across the nodes
or not). You can configure that by setting the search preference to _local: Elasticsearch Platform — Find real-time answers at scale | Elastic
.
Maybe an understanding... From the discussion on the board, I got that
a replica could be searched the same as a shard, so the idea was that by
putting one shard/replica per node, there would be no inter-node traffic
for a search. Is that wrong?
On Tue, Jan 24, 2012 at 9:35 AM, Shay Banon kimchy@gmail.com wrote:
But then you only expend on replicas, so the index will remain with a
single shard (partition of the data) size wise. Really depends on what you
are after...
OK, thanks for the help. I think having one shard and
auto-expanding replicas is the way to go since we'll be increasing the
cluster as we go forward.
On Tue, Jan 24, 2012 at 9:26 AM, Shay Banon kimchy@gmail.comwrote:
Yes, though you can't remove the setting, you can set the auto
expand setting to false (assuming you work on a live cluster and use the
index update settings API).
On Tue, Jan 24, 2012 at 5:19 PM, Kenneth Loafman < kenneth@loafman.com> wrote:
So if I remove the auto_expand_replicas, it should go back to 4?
On Tue, Jan 24, 2012 at 8:51 AM, Shay Banon kimchy@gmail.comwrote:
auto_expand_replcias is really only meaningful when you have a
single shard index.
Back to your case, with 2 shards and 1 replica, you have a total
of 4 shards, so on a 4 node cluster, you will have 1 shard per node.
The problem is that this increases the number of shards to 8 per
index. Is there a way to set number_of_shards to 1, either automatically,
or manually? Am I going about this the wrong way if I want a shard on each
node?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.