I'm running two elasticsearch nodes (0.18.6), with unicast discovery,
5 shards and 1 replica per index, 747 indices, and 2 types per index.
I restarted elasticsearch on each server (waited about a minute
between them), and checked the cluster state. It was red for 30-40
minutes, then yellow for about an hour, with the "unassigned_shards"
count slowly decreasing. At first, it was recovering 120 shards per
minute, and then it slowed to 12 shards per minute. With approximately
7400 shards, it would take hours to fully recover.
I tried running my application server (Play Framework) during the
recovery process, and found that search results would usually come
back ok, but whenever I tried to index new data, elasticsearch
wouldn't respond. I had the same results with curl -- searching
worked, but elasticsearch didn't respond when trying to add data (or
create an index).
Is it normal for elasticsearch to take so long to recover? How can I
make it faster? Is it normal for ES to not respond to PUT requests
while it's recovering?
My elasticsearch configuration file:
cluster:
name: elasticsearch
I'm running two elasticsearch nodes (0.18.6), with unicast discovery,
5 shards and 1 replica per index, 747 indices, and 2 types per index.
I restarted elasticsearch on each server (waited about a minute
between them), and checked the cluster state. It was red for 30-40
minutes, then yellow for about an hour, with the "unassigned_shards"
count slowly decreasing. At first, it was recovering 120 shards per
minute, and then it slowed to 12 shards per minute. With approximately
7400 shards, it would take hours to fully recover.
I tried running my application server (Play Framework) during the
recovery process, and found that search results would usually come
back ok, but whenever I tried to index new data, elasticsearch
wouldn't respond. I had the same results with curl -- searching
worked, but elasticsearch didn't respond when trying to add data (or
create an index).
Is it normal for elasticsearch to take so long to recover? How can I
make it faster? Is it normal for ES to not respond to PUT requests
while it's recovering?
My elasticsearch configuration file:
cluster:
name: elasticsearch
Thanks for your quick reply David! I currently have 50,000 documents.
What's an appropriate number of shards per node? I currently have an
index for each of my users, and never need to search more than one
user's data at a time. Would it be better to have just one index and
use a filter to limit results to one user (each document has a user_id
field)? Would you recommend fewer (or more, if I reduce the number of
indices) shards per index? I see at
that the number of shards per index affects how many documents I can
store -- do you know approximately how many documents each shard can
handle?
I'm running two elasticsearch nodes (0.18.6), with unicast discovery,
5 shards and 1 replica per index, 747 indices, and 2 types per index.
I restarted elasticsearch on each server (waited about a minute
between them), and checked the cluster state. It was red for 30-40
minutes, then yellow for about an hour, with the "unassigned_shards"
count slowly decreasing. At first, it was recovering 120 shards per
minute, and then it slowed to 12 shards per minute. With approximately
7400 shards, it would take hours to fully recover.
I tried running my application server (Play Framework) during the
recovery process, and found that search results would usually come
back ok, but whenever I tried to index new data, elasticsearch
wouldn't respond. I had the same results with curl -- searching
worked, but elasticsearch didn't respond when trying to add data (or
create an index).
Is it normal for elasticsearch to take so long to recover? How can I
make it faster? Is it normal for ES to not respond to PUT requests
while it's recovering?
My elasticsearch configuration file:
cluster:
name: elasticsearch
Maybe the _close and _open of index could help you out. Just close all
indices and open them just before you're going to search them. Close them
again after a certain time of idleness.
Disclaimer: The information contained in this message and attachments is
intended solely for the attention and use of the named addressee and may be
confidential. If you are not the intended recipient, you are reminded that
the information remains the property of the sender. You must not use,
disclose, distribute, copy, print or rely on this e-mail. If you have
received this message in error, please contact the sender immediately and
irrevocably delete this message and any copies.
Thanks for your quick reply David! I currently have 50,000 documents.
What's an appropriate number of shards per node? I currently have an
index for each of my users, and never need to search more than one
user's data at a time. Would it be better to have just one index and
use a filter to limit results to one user (each document has a user_id
field)? Would you recommend fewer (or more, if I reduce the number of
indices) shards per index? I see at Elasticsearch Platform — Find real-time answers at scale | Elastic
that the number of shards per index affects how many documents I can
store -- do you know approximately how many documents each shard can
handle?
I'm running two elasticsearch nodes (0.18.6), with unicast discovery,
5 shards and 1 replica per index, 747 indices, and 2 types per index.
I restarted elasticsearch on each server (waited about a minute
between them), and checked the cluster state. It was red for 30-40
minutes, then yellow for about an hour, with the "unassigned_shards"
count slowly decreasing. At first, it was recovering 120 shards per
minute, and then it slowed to 12 shards per minute. With approximately
7400 shards, it would take hours to fully recover.
I tried running my application server (Play Framework) during the
recovery process, and found that search results would usually come
back ok, but whenever I tried to index new data, elasticsearch
wouldn't respond. I had the same results with curl -- searching
worked, but elasticsearch didn't respond when trying to add data (or
create an index).
Is it normal for elasticsearch to take so long to recover? How can I
make it faster? Is it normal for ES to not respond to PUT requests
while it's recovering?
My elasticsearch configuration file:
cluster:
name: elasticsearch
On Fri, 2012-09-21 at 10:56 -0700, David Loehr wrote:
Thanks for your quick reply David! I currently have 50,000 documents.
What's an appropriate number of shards per node? I currently have an
index for each of my users, and never need to search more than one
user's data at a time. Would it be better to have just one index and
use a filter to limit results to one user (each document has a user_id
field)? Would you recommend fewer (or more, if I reduce the number of
indices) shards per index? I see at Elasticsearch Platform — Find real-time answers at scale | Elastic
that the number of shards per index affects how many documents I can
store -- do you know approximately how many documents each shard can
handle?
Have a look at Shay's talk about scaling strategies. It'll be very
useful to you
On Friday, September 21, 2012 1:16:18 PM UTC-4, David Loehr wrote:
I'm running two elasticsearch nodes (0.18.6), with unicast discovery,
5 shards and 1 replica per index, 747 indices, and 2 types per index.
I restarted elasticsearch on each server (waited about a minute
between them), and checked the cluster state. It was red for 30-40
minutes, then yellow for about an hour, with the "unassigned_shards"
count slowly decreasing. At first, it was recovering 120 shards per
minute, and then it slowed to 12 shards per minute. With approximately
7400 shards, it would take hours to fully recover.
I tried running my application server (Play Framework) during the
recovery process, and found that search results would usually come
back ok, but whenever I tried to index new data, elasticsearch
wouldn't respond. I had the same results with curl -- searching
worked, but elasticsearch didn't respond when trying to add data (or
create an index).
Is it normal for elasticsearch to take so long to recover? How can I
make it faster? Is it normal for ES to not respond to PUT requests
while it's recovering?
My elasticsearch configuration file:
cluster:
name: elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
