Cluster misconfigured or elastic is just SLOW!?

Elastic Stack Elasticsearch
1

Hi,
I am evaluating elastic search for a data warehouse project. I have a
9 node cluster with 1 replica. I have loaded 6 days worth of data.
Things seem sluggish all around. Here is the health of the nodes

{
"active_primary_shards": 9251,
"active_shards": 13509,
"cluster_name": "qa-elasticsearch",
"initializing_shards": 23,
"number_of_data_nodes": 9,
"number_of_nodes": 9,
"relocating_shards": 0,
"status": "red",
"timed_out": false,
"unassigned_shards": 5054
}

I am wondering if I am missing some configuration that will speed things up
or if I did something wrong? I see the status is "RED" and it seems that
there are still node initializing and the cluster has been up for over 24
hours. I used logstash to push the data into elastic.
Any help would be appreciated....or if any additional information is needed
I can provide.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a1c4cb64-82f5-4ab1-be6a-9b29e0a01be9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Yes you have something very wrong. That is showing you have a huge number
of shards and the cluster is obviously struggling to allocate all of them.
You said you have 9 nodes and 1 replica but you didn't specify how many
shards per index?

Kimbro

On Thu, Jan 22, 2015 at 11:45 AM, Sam Flint sam.flint@magnetic.com wrote:

Hi,
I am evaluating Elasticsearch for a data warehouse project. I have a
9 node cluster with 1 replica. I have loaded 6 days worth of data.
Things seem sluggish all around. Here is the health of the nodes

{
"active_primary_shards": 9251,
"active_shards": 13509,
"cluster_name": "qa-elasticsearch",
"initializing_shards": 23,
"number_of_data_nodes": 9,
"number_of_nodes": 9,
"relocating_shards": 0,
"status": "red",
"timed_out": false,
"unassigned_shards": 5054
}

I am wondering if I am missing some configuration that will speed things
up or if I did something wrong? I see the status is "RED" and it seems
that there are still node initializing and the cluster has been up for over
24 hours. I used logstash to push the data into elastic.
Any help would be appreciated....or if any additional information is
needed I can provide.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1c4cb64-82f5-4ab1-be6a-9b29e0a01be9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a1c4cb64-82f5-4ab1-be6a-9b29e0a01be9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAA0DmXbpxtS81ogsy2ZjQ1sabRoWa4B%2BLUMt7FerPR2Y70FWNg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

That's a massive number of shards! As Kimbro pointed out, that's likely
your problem.

On 23 January 2015 at 06:04, Kimbro Staken kstaken@kstaken.com wrote:

Yes you have something very wrong. That is showing you have a huge number
of shards and the cluster is obviously struggling to allocate all of them.
You said you have 9 nodes and 1 replica but you didn't specify how many
shards per index?

Kimbro

On Thu, Jan 22, 2015 at 11:45 AM, Sam Flint sam.flint@magnetic.com
wrote:

Hi,
I am evaluating Elasticsearch for a data warehouse project. I have
a 9 node cluster with 1 replica. I have loaded 6 days worth of data.
Things seem sluggish all around. Here is the health of the nodes

{
"active_primary_shards": 9251,
"active_shards": 13509,
"cluster_name": "qa-elasticsearch",
"initializing_shards": 23,
"number_of_data_nodes": 9,
"number_of_nodes": 9,
"relocating_shards": 0,
"status": "red",
"timed_out": false,
"unassigned_shards": 5054
}

I am wondering if I am missing some configuration that will speed things
up or if I did something wrong? I see the status is "RED" and it seems
that there are still node initializing and the cluster has been up for over
24 hours. I used logstash to push the data into elastic.
Any help would be appreciated....or if any additional information is
needed I can provide.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a1c4cb64-82f5-4ab1-be6a-9b29e0a01be9%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/a1c4cb64-82f5-4ab1-be6a-9b29e0a01be9%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAA0DmXbpxtS81ogsy2ZjQ1sabRoWa4B%2BLUMt7FerPR2Y70FWNg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAA0DmXbpxtS81ogsy2ZjQ1sabRoWa4B%2BLUMt7FerPR2Y70FWNg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_PVuWo4Y5J%2BcVr4Fy9f8iQnvd8xtt-%3D9v10Q-wnVVYEg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Here is the .yml part for the shards

Set the number of shards (splits) of an index (5 by default):

index.number_of_shards: 64

Set the number of replicas (additional copies) of an index (1 by default):

index.number_of_replicas: 1

I have it set to 64 shards and 1 replica. Is there a recommended amount
of shards that I should be using?

On Thursday, January 22, 2015 at 1:45:23 PM UTC-5, Sam Flint wrote:

Hi,
I am evaluating Elasticsearch for a data warehouse project. I have a
9 node cluster with 1 replica. I have loaded 6 days worth of data.
Things seem sluggish all around. Here is the health of the nodes

{
"active_primary_shards": 9251,
"active_shards": 13509,
"cluster_name": "qa-elasticsearch",
"initializing_shards": 23,
"number_of_data_nodes": 9,
"number_of_nodes": 9,
"relocating_shards": 0,
"status": "red",
"timed_out": false,
"unassigned_shards": 5054
}

I am wondering if I am missing some configuration that will speed things
up or if I did something wrong? I see the status is "RED" and it seems
that there are still node initializing and the cluster has been up for over
24 hours. I used logstash to push the data into elastic.
Any help would be appreciated....or if any additional information is
needed I can provide.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/244fbf99-5ec4-4052-a860-16fad53159ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

How many shards to use is a complicated question and depends on the
specific use case. For testing in this scenario though, it's likely that
just matching the number of nodes you have would be a good choice. Then you
will have 1 primary shard for each index on each node.

That said it also looks like maybe you're creating more indices than is
ideal as well. 64 shards with 1 replica and 6 days data in daily indices
should only produce 768 total shards. Looks to me like maybe you have 1
index per hour?

On Fri, Jan 23, 2015 at 8:28 AM, Sam Flint sam.flint@magnetic.com wrote:

Here is the .yml part for the shards

Set the number of shards (splits) of an index (5 by default):

index.number_of_shards: 64

Set the number of replicas (additional copies) of an index (1 by

default):

index.number_of_replicas: 1

I have it set to 64 shards and 1 replica. Is there a recommended amount
of shards that I should be using?

On Thursday, January 22, 2015 at 1:45:23 PM UTC-5, Sam Flint wrote:

Hi,
I am evaluating Elasticsearch for a data warehouse project. I have
a 9 node cluster with 1 replica. I have loaded 6 days worth of data.
Things seem sluggish all around. Here is the health of the nodes

{
"active_primary_shards": 9251,
"active_shards": 13509,
"cluster_name": "qa-elasticsearch",
"initializing_shards": 23,
"number_of_data_nodes": 9,
"number_of_nodes": 9,
"relocating_shards": 0,
"status": "red",
"timed_out": false,
"unassigned_shards": 5054
}

I am wondering if I am missing some configuration that will speed things
up or if I did something wrong? I see the status is "RED" and it seems
that there are still node initializing and the cluster has been up for over
24 hours. I used logstash to push the data into elastic.
Any help would be appreciated....or if any additional information is
needed I can provide.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/244fbf99-5ec4-4052-a860-16fad53159ed%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/244fbf99-5ec4-4052-a860-16fad53159ed%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAA0DmXbZRa-_05-eKuF7Tej88q4u1gtK4RB9b2ypTDn1gfacBw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.