I am not realizing how to get the default number of shards per node or the
maximun one, because I think I am having 'red' status since I have more
than 100 shards in one index, and each index has in average 200K docs.
On Friday, November 15, 2013 8:07:45 AM UTC-8, @mromagnoli wrote:
I am not realizing how to get the default number of shards per node or the
maximun one, because I think I am having 'red' status since I have more
than 100 shards in one index, and each index has in average 200K docs.
Thanks in advance!
Sorry you're having issues! To the best of my knowledge, there's no maximum
number of shards for a single server; if anything, it would be limited by
memory available.
Red means some shards can't be loaded, but this can be caused by corrupt or
missing shard data. Do you have any elasticsearch logs that might help here?
On Friday, November 15, 2013 10:07:18 AM UTC-8, Jordan Sissel wrote:
On Friday, November 15, 2013 8:07:45 AM UTC-8, @mromagnoli wrote:
I am not realizing how to get the default number of shards per node or
the maximun one, because I think I am having 'red' status since I have more
than 100 shards in one index, and each index has in average 200K docs.
Thanks in advance!
Sorry you're having issues! To the best of my knowledge, there's no
maximum number of shards for a single server; if anything, it would be
limited by memory available.
Red means some shards can't be loaded, but this can be caused by corrupt
or missing shard data. Do you have any elasticsearch logs that might help
here?
-Jordan
One additional note I forgot to mention - file count limits could be a
cause of your problems here.
Pretty much all linux distros ship with a default 'open file' limit of
1024. This is a really bad default for elasticsearch, especially if you
have many shards. You can see file-count related errors in ES's logs for
things like "Failed to create shard" for example:
Failed to create shard, message [IndexShardCreationException[[exampl
e][1691] failed to create shard]; nested: IOException[directory
'/mnt/btrfs/data/ela
sticsearch/nodes/0/indices/example/1691/index' exists and is a directory,
but cannot
be listed: list() returned null]; ]]
You can find more details about checking this file count limit here:
On Friday, November 15, 2013 10:07:18 AM UTC-8, Jordan Sissel wrote:
On Friday, November 15, 2013 8:07:45 AM UTC-8, @mromagnoli wrote:
I am not realizing how to get the default number of shards per node or the maximun one, because I think I am having 'red' status since I have more than 100 shards in one index, and each index has in average 200K docs.
Thanks in advance!
Sorry you're having issues! To the best of my knowledge, there's no maximum number of shards for a single server; if anything, it would be limited by memory available.
Red means some shards can't be loaded, but this can be caused by corrupt or missing shard data. Do you have any elasticsearch logs that might help here?
-Jordan
One additional note I forgot to mention - file count limits could be a cause of your problems here.
Pretty much all linux distros ship with a default 'open file' limit of 1024. This is a really bad default for elasticsearch, especially if you have many shards. You can see file-count related errors in ES's logs for things like "Failed to create shard" for example:
Failed to create shard, message [IndexShardCreationException[[exampl
e][1691] failed to create shard]; nested: IOException[directory '/mnt/btrfs/data/ela
sticsearch/nodes/0/indices/example/1691/index' exists and is a directory, but cannot
be listed: list() returned null]; ]]
Thanks for reply, Jordan. We have already set the number of the max files
for user elasticsearch to 65535 so we think is a good number. In the other
hand, we are seeing this in our log:
[netty.channel.DefaultChannelPipeline] An exception was thrown by an
exception handler.
java.util.concurrent.RejectedExecutionException: Worker has already been
shutdown
[151089] Failed to execute fetch phase
org.elasticsearch.search.SearchContextMissingException: No search context
found for id [151089]
El viernes, 15 de noviembre de 2013 13:07:45 UTC-3, @mromagnoli escribió:
I am not realizing how to get the default number of shards per node or the
maximun one, because I think I am having 'red' status since I have more
than 100 shards in one index, and each index has in average 200K docs.
On Friday, November 15, 2013 10:07:18 AM UTC-8, Jordan Sissel wrote:
On Friday, November 15, 2013 8:07:45 AM UTC-8, @mromagnoli wrote:
I am not realizing how to get the default number of shards per node or
the maximun one, because I think I am having 'red' status since I have more
than 100 shards in one index, and each index has in average 200K docs.
Thanks in advance!
Sorry you're having issues! To the best of my knowledge, there's no
maximum number of shards for a single server; if anything, it would be
limited by memory available.
Red means some shards can't be loaded, but this can be caused by corrupt
or missing shard data. Do you have any elasticsearch logs that might help
here?
-Jordan
One additional note I forgot to mention - file count limits could be a
cause of your problems here.
Pretty much all linux distros ship with a default 'open file' limit of
1024. This is a really bad default for elasticsearch, especially if you
have many shards. You can see file-count related errors in ES's logs for
things like "Failed to create shard" for example:
Failed to create shard, message [IndexShardCreationException[[exampl
e][1691] failed to create shard]; nested: IOException[directory
'/mnt/btrfs/data/ela
sticsearch/nodes/0/indices/example/1691/index' exists and is a directory,
but cannot
be listed: list() returned null]; ]]
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
