# Unassigned\_shards problam
**URL:** https://discuss.elastic.co/t/unassigned-shards-problam/14544
**Category:** Elasticsearch
**Created:** [November 22, 2013, 12:35pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544 "2013-11-22T12:35:05Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 22, 2013, 12:35pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/1 "2013-11-22T12:35:05Z")
so..
I decided to create a three nodes cluster (two data nodes and one arbiter)
All three nodes are using ec2 discovery
but when i'm calling /\_cluster/health on the _master node_ I get the
following response:
{
- cluster\_name: "SCluster",
- status: "red",
- timed\_out: false,
- number\_of\_nodes: 1,
- number\_of\_data\_nodes: 1,
- active\_primary\_shards: 0,
- active\_shards: 0,
- relocating\_shards: 0,
- initializing\_shards: 0,
- unassigned\_shards: 48
}
I can detect two troubling parameters:
- active\_primary\_shards: 0
- AND
- unassigned\_shards: 48
-
another troubling issue is that the other data node and the arbiter
responding
- status: "green"
- witch makes me belive that they are not really connected to the same
cluster..
-
-
-
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [November 22, 2013, 12:39pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/2 "2013-11-22T12:39:42Z")
Hi Daniel,
It looks like you nodes do not discover each other - that number of nodes
should be 3 for you.
I'd look at the logs for clues of why it fails.
Cheers,
Boaz
On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> so..
> I decided to create a three nodes cluster (two data nodes and one arbiter)
> All three nodes are using ec2 discovery
>
> but when i'm calling /\_cluster/health on the _master node_ I get the
> following response:
>
> {
>
> - cluster\_name: "SCluster",
> - status: "red",
> - timed\_out: false,
> - number\_of\_nodes: 1,
> - number\_of\_data\_nodes: 1,
> - active\_primary\_shards: 0,
> - active\_shards: 0,
> - relocating\_shards: 0,
> - initializing\_shards: 0,
> - unassigned\_shards: 48
>
> }
>
> I can detect two troubling parameters:
>
> - active\_primary\_shards: 0
> - AND
> - unassigned\_shards: 48
> -
>
> another troubling issue is that the other data node and the arbiter
> responding
>
> - status: "green"
>
> - witch makes me belive that they are not really connected to the same
> cluster..
>
> -
>
> -
> -
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 22, 2013, 12:59pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/3 "2013-11-22T12:59:25Z")
ok.. thats embarrassing.. i cant locate the log file in
"/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> Hi Daniel,
>
> It looks like you nodes do not discover each other - that number of nodes
> should be 3 for you.
>
> I'd look at the logs for clues of why it fails.
>
> Cheers,
> Boaz
>
> On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
>
> > so..
> > I decided to create a three nodes cluster (two data nodes and one arbiter)
> > All three nodes are using ec2 discovery
> >
> > but when i'm calling /\_cluster/health on the _master node_ I get the
> > following response:
> >
> > {
> >
> > - cluster\_name: "SCluster",
> > - status: "red",
> > - timed\_out: false,
> > - number\_of\_nodes: 1,
> > - number\_of\_data\_nodes: 1,
> > - active\_primary\_shards: 0,
> > - active\_shards: 0,
> > - relocating\_shards: 0,
> > - initializing\_shards: 0,
> > - unassigned\_shards: 48
> >
> > }
> >
> > I can detect two troubling parameters:
> >
> > - active\_primary\_shards: 0
> > - AND
> > - unassigned\_shards: 48
> > -
> >
> > another troubling issue is that the other data node and the arbiter
> > responding
> >
> > - status: "green"
> >
> > - witch makes me belive that they are not really connected to the
> > same cluster..
> >
> > -
> >
> > -
> > -
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [November 22, 2013, 1:53pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/4 "2013-11-22T13:53:16Z")
Do you use the rpm/debia packages or is it an install from a tar ball/ zip?
Can you share the elasticsearch.yml configuration? Also, this can
potentially be overridden using command line parameters (but I assume you
don't do that).
On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com) wrote:
> ok.. thats embarrassing.. i cant locate the log file in
> "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
>
> On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
>
> > Hi Daniel,
> >
> > It looks like you nodes do not discover each other - that number of nodes
> > should be 3 for you.
> >
> > I'd look at the logs for clues of why it fails.
> >
> > Cheers,
> > Boaz
> >
> > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> >
> > > so..
> > > I decided to create a three nodes cluster (two data nodes and one
> > > arbiter)
> > > All three nodes are using ec2 discovery
> > >
> > > but when i'm calling /\_cluster/health on the _master node_ I get the
> > > following response:
> > >
> > > {
> > >
> > > - cluster\_name: "SCluster",
> > > - status: "red",
> > > - timed\_out: false,
> > > - number\_of\_nodes: 1,
> > > - number\_of\_data\_nodes: 1,
> > > - active\_primary\_shards: 0,
> > > - active\_shards: 0,
> > > - relocating\_shards: 0,
> > > - initializing\_shards: 0,
> > > - unassigned\_shards: 48
> > >
> > > }
> > >
> > > I can detect two troubling parameters:
> > >
> > > - active\_primary\_shards: 0
> > > - AND
> > > - unassigned\_shards: 48
> > > -
> > >
> > > another troubling issue is that the other data node and the arbiter
> > > responding
> > >
> > > - status: "green"
> > >
> > > - witch makes me belive that they are not really connected to the
> > > same cluster..
> > >
> > > -
> > >
> > > -
> > > -
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 22, 2013, 3:47pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/5 "2013-11-22T15:47:23Z")
I installed from tar.
my configuration.yml looks like this:
cluster.name: SCluster
node.name: "SCNode1Master"
node.master: true
index.number\_of\_shards: 8
index.number\_of\_replicas: 2
path:
logs: /var/log/elasticsearch
cloud:
aws:
access\_key: XXXXXX
secret\_key: XXXXXXXXX
discovery:
type: ec2
ec2:
groups: GROUP\_NAME
cloud.node.auto\_attributes: true
On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> Do you use the rpm/debia packages or is it an install from a tar ball/
> zip? Can you share the elasticsearch.yml configuration? Also, this can
> potentially be overridden using command line parameters (but I assume you
> don't do that).
>
> On Fri, Nov 22, 2013 at 1:59 PM, DanielR \<[danielr...@gmail.com](mailto:danielr...@gmail.com)\
>
> > wrote:
>
> > ok.. thats embarrassing.. i cant locate the log file in
> > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> >
> > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> >
> > > Hi Daniel,
> > >
> > > It looks like you nodes do not discover each other - that number of
> > > nodes should be 3 for you.
> > >
> > > I'd look at the logs for clues of why it fails.
> > >
> > > Cheers,
> > > Boaz
> > >
> > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > >
> > > > so..
> > > > I decided to create a three nodes cluster (two data nodes and one
> > > > arbiter)
> > > > All three nodes are using ec2 discovery
> > > >
> > > > but when i'm calling /\_cluster/health on the _master node_ I get the
> > > > following response:
> > > >
> > > > {
> > > >
> > > > - cluster\_name: "SCluster",
> > > > - status: "red",
> > > > - timed\_out: false,
> > > > - number\_of\_nodes: 1,
> > > > - number\_of\_data\_nodes: 1,
> > > > - active\_primary\_shards: 0,
> > > > - active\_shards: 0,
> > > > - relocating\_shards: 0,
> > > > - initializing\_shards: 0,
> > > > - unassigned\_shards: 48
> > > >
> > > > }
> > > >
> > > > I can detect two troubling parameters:
> > > >
> > > > - active\_primary\_shards: 0
> > > > - AND
> > > > - unassigned\_shards: 48
> > > > -
> > > >
> > > > another troubling issue is that the other data node and the arbiter
> > > > responding
> > > >
> > > > - status: "green"
> > > >
> > > > - witch makes me belive that they are not really connected to the
> > > > same cluster..
> > > >
> > > > -
> > > >
> > > > -
> > > > -
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit
> > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [November 25, 2013, 1:07pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/6 "2013-11-25T13:07:35Z")
Hi Daniel,
Just double checking - you do have space on the logs:
/var/log/elasticsearch? Also check for a logs folder where you untared ES
into.
Cheers,
Boaz
On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> I installed from tar.
>
> my configuration.yml looks like this:
>
> cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> SCluster
> node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> "SCNode1Master"
>
> node.master: true
> index.number\_of\_shards: 8
>
> index.number\_of\_replicas: 2
> path:
> logs: /var/log/elasticsearch
>
> cloud:
> aws:
> access\_key: XXXXXX
> secret\_key: XXXXXXXXX
> discovery:
> type: ec2
> ec2:
> groups: GROUP\_NAME
>
> cloud.node.auto\_attributes: true
>
> On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
>
> > Do you use the rpm/debia packages or is it an install from a tar ball/
> > zip? Can you share the elasticsearch.yml configuration? Also, this can
> > potentially be overridden using command line parameters (but I assume you
> > don't do that).
> >
> > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> >
> > > ok.. thats embarrassing.. i cant locate the log file in
> > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > >
> > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > >
> > > > Hi Daniel,
> > > >
> > > > It looks like you nodes do not discover each other - that number of
> > > > nodes should be 3 for you.
> > > >
> > > > I'd look at the logs for clues of why it fails.
> > > >
> > > > Cheers,
> > > > Boaz
> > > >
> > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > >
> > > > > so..
> > > > > I decided to create a three nodes cluster (two data nodes and one
> > > > > arbiter)
> > > > > All three nodes are using ec2 discovery
> > > > >
> > > > > but when i'm calling /\_cluster/health on the _master node_ I get the
> > > > > following response:
> > > > >
> > > > > {
> > > > >
> > > > > - cluster\_name: "SCluster",
> > > > > - status: "red",
> > > > > - timed\_out: false,
> > > > > - number\_of\_nodes: 1,
> > > > > - number\_of\_data\_nodes: 1,
> > > > > - active\_primary\_shards: 0,
> > > > > - active\_shards: 0,
> > > > > - relocating\_shards: 0,
> > > > > - initializing\_shards: 0,
> > > > > - unassigned\_shards: 48
> > > > >
> > > > > }
> > > > >
> > > > > I can detect two troubling parameters:
> > > > >
> > > > > - active\_primary\_shards: 0
> > > > > - AND
> > > > > - unassigned\_shards: 48
> > > > > -
> > > > >
> > > > > another troubling issue is that the other data node and the arbiter
> > > > > responding
> > > > >
> > > > > - status: "green"
> > > > >
> > > > > - witch makes me belive that they are not really connected to the
> > > > > same cluster..
> > > > >
> > > > > -
> > > > >
> > > > > -
> > > > > -
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to a topic in the
> > > > > Google Groups "elasticsearch" group.
> > > > > To unsubscribe from this topic, visit
> > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 25, 2013, 10:51pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/7 "2013-11-25T22:51:17Z")
ok 🙂
found the logs
I think maybe cluster.service is trying to find the other nodes on port
9300...
so.. here it is:
[2013-11-25 21:56:08,072][INFO][node] [SCNode1Master]
{0.90.1}[926]: initializing ...
[2013-11-25 21:56:08,196][INFO][plugins] [SCNode1Master]
loaded [cloud-aws], sites
[2013-11-25 21:56:13,667][INFO][node] [SCNode1Master]
{0.90.1}[926]: initialized
[2013-11-25 21:56:13,667][INFO][node] [SCNode1Master]
{0.90.1}[926]: starting ...
[2013-11-25 21:56:13,849][INFO][transport] [SCNode1Master]
bound\_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish\_address
{inet[/:9300]}
[2013-11-25 21:56:16,962][INFO][cluster.service] [SCNode1Master]
new\_master [SCNode1Master][\][inet[/ter=true}, reason:
zen-disco-join (elected\_as\_master)
[2013-11-25 21:56:17,014][INFO][discovery] [SCNode1Master]
SCluster/\
[2013-11-25 21:56:17,053][INFO][http] [SCNode1Master]
bound\_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish\_address
{inet[/:9200]}
[2013-11-25 21:56:17,053][INFO][node] [SCNode1Master]
{0.90.1}[926]: started
[2013-11-25 21:56:17,114][INFO][gateway] [SCNode1Master]
recovered [2] indices into cluster\_state
~
On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> Hi Daniel,
>
> Just double checking - you do have space on the logs:
> /var/log/elasticsearch? Also check for a logs folder where you untared ES
> into.
>
> Cheers,
> Boaz
>
> On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
>
> > I installed from tar.
> >
> > my configuration.yml looks like this:
> >
> > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > SCluster
> > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > "SCNode1Master"
> >
> > node.master: true
> > index.number\_of\_shards: 8
> >
> > index.number\_of\_replicas: 2
> > path:
> > logs: /var/log/elasticsearch
> >
> > cloud:
> > aws:
> > access\_key: XXXXXX
> > secret\_key: XXXXXXXXX
> > discovery:
> > type: ec2
> > ec2:
> > groups: GROUP\_NAME
> >
> > cloud.node.auto\_attributes: true
> >
> > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> >
> > > Do you use the rpm/debia packages or is it an install from a tar ball/
> > > zip? Can you share the elasticsearch.yml configuration? Also, this can
> > > potentially be overridden using command line parameters (but I assume you
> > > don't do that).
> > >
> > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > >
> > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > >
> > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > >
> > > > > Hi Daniel,
> > > > >
> > > > > It looks like you nodes do not discover each other - that number of
> > > > > nodes should be 3 for you.
> > > > >
> > > > > I'd look at the logs for clues of why it fails.
> > > > >
> > > > > Cheers,
> > > > > Boaz
> > > > >
> > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > >
> > > > > > so..
> > > > > > I decided to create a three nodes cluster (two data nodes and one
> > > > > > arbiter)
> > > > > > All three nodes are using ec2 discovery
> > > > > >
> > > > > > but when i'm calling /\_cluster/health on the _master node_ I get the
> > > > > > following response:
> > > > > >
> > > > > > {
> > > > > >
> > > > > > - cluster\_name: "SCluster",
> > > > > > - status: "red",
> > > > > > - timed\_out: false,
> > > > > > - number\_of\_nodes: 1,
> > > > > > - number\_of\_data\_nodes: 1,
> > > > > > - active\_primary\_shards: 0,
> > > > > > - active\_shards: 0,
> > > > > > - relocating\_shards: 0,
> > > > > > - initializing\_shards: 0,
> > > > > > - unassigned\_shards: 48
> > > > > >
> > > > > > }
> > > > > >
> > > > > > I can detect two troubling parameters:
> > > > > >
> > > > > > - active\_primary\_shards: 0
> > > > > > - AND
> > > > > > - unassigned\_shards: 48
> > > > > > -
> > > > > >
> > > > > > another troubling issue is that the other data node and the arbiter
> > > > > > responding
> > > > > >
> > > > > > - status: "green"
> > > > > >
> > > > > > - witch makes me belive that they are not really connected to the
> > > > > > same cluster..
> > > > > >
> > > > > > -
> > > > > >
> > > > > > -
> > > > > > -
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to a topic in the
> > > > > > Google Groups "elasticsearch" group.
> > > > > > To unsubscribe from this topic, visit
> > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe)
> > > > > > .
> > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 25, 2013, 10:53pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/8 "2013-11-25T22:53:59Z")
9300 is the port ES uses for intercluster comms, so that's fine.
Browse through the logs on each of your nodes and see if you can find
references to the others joining the master.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)
web: [www.campaignmonitor.com](http://www.campaignmonitor.com)
On 26 November 2013 09:51, DanielR [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com) wrote:
> ok 🙂
> found the logs
> I think maybe cluster.service is trying to find the other nodes on port
> 9300...
>
> so.. here it is:
>
> [2013-11-25 21:56:08,072][INFO][node]
> [SCNode1Master] {0.90.1}[926]: initializing ...
> [2013-11-25 21:56:08,196][INFO][plugins]
> [SCNode1Master] loaded [cloud-aws], sites
> [2013-11-25 21:56:13,667][INFO][node]
> [SCNode1Master] {0.90.1}[926]: initialized
> [2013-11-25 21:56:13,667][INFO][node]
> [SCNode1Master] {0.90.1}[926]: starting ...
> [2013-11-25 21:56:13,849][INFO][transport]
> [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> publish\_address {inet[/:9300]}
> [2013-11-25 21:56:16,962][INFO][cluster.service]
> [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> reason: zen-disco-join (elected\_as\_master)
> [2013-11-25 21:56:17,014][INFO][discovery]
> [SCNode1Master] SCluster/\
> [2013-11-25 21:56:17,053][INFO][http]
> [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> publish\_address {inet[/:9200]}
> [2013-11-25 21:56:17,053][INFO][node]
> [SCNode1Master] {0.90.1}[926]: started
> [2013-11-25 21:56:17,114][INFO][gateway]
> [SCNode1Master] recovered [2] indices into cluster\_state
> ~
>
> On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
>
> > Hi Daniel,
> >
> > Just double checking - you do have space on the logs:
> > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > into.
> >
> > Cheers,
> > Boaz
> >
> > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> >
> > > I installed from tar.
> > >
> > > my configuration.yml looks like this:
> > >
> > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > SCluster
> > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > "SCNode1Master"
> > >
> > > node.master: true
> > > index.number\_of\_shards: 8
> > >
> > > index.number\_of\_replicas: 2
> > > path:
> > > logs: /var/log/elasticsearch
> > >
> > > cloud:
> > > aws:
> > > access\_key: XXXXXX
> > > secret\_key: XXXXXXXXX
> > > discovery:
> > > type: ec2
> > > ec2:
> > > groups: GROUP\_NAME
> > >
> > > cloud.node.auto\_attributes: true
> > >
> > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > >
> > > > Do you use the rpm/debia packages or is it an install from a tar ball/
> > > > zip? Can you share the elasticsearch.yml configuration? Also, this can
> > > > potentially be overridden using command line parameters (but I assume you
> > > > don't do that).
> > > >
> > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > >
> > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > >
> > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > >
> > > > > > Hi Daniel,
> > > > > >
> > > > > > It looks like you nodes do not discover each other - that number of
> > > > > > nodes should be 3 for you.
> > > > > >
> > > > > > I'd look at the logs for clues of why it fails.
> > > > > >
> > > > > > Cheers,
> > > > > > Boaz
> > > > > >
> > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > >
> > > > > > > so..
> > > > > > > I decided to create a three nodes cluster (two data nodes and one
> > > > > > > arbiter)
> > > > > > > All three nodes are using ec2 discovery
> > > > > > >
> > > > > > > but when i'm calling /\_cluster/health on the _master node_ I get
> > > > > > > the following response:
> > > > > > >
> > > > > > > {
> > > > > > >
> > > > > > > - cluster\_name: "SCluster",
> > > > > > > - status: "red",
> > > > > > > - timed\_out: false,
> > > > > > > - number\_of\_nodes: 1,
> > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > - active\_primary\_shards: 0,
> > > > > > > - active\_shards: 0,
> > > > > > > - relocating\_shards: 0,
> > > > > > > - initializing\_shards: 0,
> > > > > > > - unassigned\_shards: 48
> > > > > > >
> > > > > > > }
> > > > > > >
> > > > > > > I can detect two troubling parameters:
> > > > > > >
> > > > > > > - active\_primary\_shards: 0
> > > > > > > - AND
> > > > > > > - unassigned\_shards: 48
> > > > > > > -
> > > > > > >
> > > > > > > another troubling issue is that the other data node and the arbiter
> > > > > > > responding
> > > > > > >
> > > > > > > - status: "green"
> > > > > > >
> > > > > > > - witch makes me belive that they are not really connected to
> > > > > > > the same cluster..
> > > > > > >
> > > > > > > -
> > > > > > >
> > > > > > > -
> > > > > > > -
> > > > > > >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to a topic in the
> > > > > > > Google Groups "elasticsearch" group.
> > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > > topic/elasticsearch/6cYLWYw6XHg/unsubscribe.
> > > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google Groups
> > > > "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send an
> > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 25, 2013, 10:57pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/9 "2013-11-25T22:57:20Z")
Nope...
Both the arbiter and the second node are flagged with "new\_master"
[2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
new\_master [Node3][\][inet[/:9300]], reason: zen-disco-join
(elected\_as\_master)
[2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
new\_master [Arbiter][\][inet[/:9300]], reason: zen-disco-join
(elected\_as\_master)
On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> 9300 is the port ES uses for intercluster comms, so that's fine.
>
> Browse through the logs on each of your nodes and see if you can find
> references to the others joining the master.
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \
> web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
>
> On 26 November 2013 09:51, DanielR \<[danielr...@gmail.com](mailto:danielr...@gmail.com) \\>wrote:
>
> > ok 🙂
> > found the logs
> > I think maybe cluster.service is trying to find the other nodes on port
> > 9300...
> >
> > so.. here it is:
> >
> > [2013-11-25 21:56:08,072][INFO][node]
> > [SCNode1Master] {0.90.1}[926]: initializing ...
> > [2013-11-25 21:56:08,196][INFO][plugins]
> > [SCNode1Master] loaded [cloud-aws], sites
> > [2013-11-25 21:56:13,667][INFO][node]
> > [SCNode1Master] {0.90.1}[926]: initialized
> > [2013-11-25 21:56:13,667][INFO][node]
> > [SCNode1Master] {0.90.1}[926]: starting ...
> > [2013-11-25 21:56:13,849][INFO][transport]
> > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > publish\_address {inet[/:9300]}
> > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > reason: zen-disco-join (elected\_as\_master)
> > [2013-11-25 21:56:17,014][INFO][discovery]
> > [SCNode1Master] SCluster/\
> > [2013-11-25 21:56:17,053][INFO][http]
> > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > publish\_address {inet[/:9200]}
> > [2013-11-25 21:56:17,053][INFO][node]
> > [SCNode1Master] {0.90.1}[926]: started
> > [2013-11-25 21:56:17,114][INFO][gateway]
> > [SCNode1Master] recovered [2] indices into cluster\_state
> > ~
> >
> > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> >
> > > Hi Daniel,
> > >
> > > Just double checking - you do have space on the logs:
> > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > into.
> > >
> > > Cheers,
> > > Boaz
> > >
> > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > >
> > > > I installed from tar.
> > > >
> > > > my configuration.yml looks like this:
> > > >
> > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > SCluster
> > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > "SCNode1Master"
> > > >
> > > > node.master: true
> > > > index.number\_of\_shards: 8
> > > >
> > > > index.number\_of\_replicas: 2
> > > > path:
> > > > logs: /var/log/elasticsearch
> > > >
> > > > cloud:
> > > > aws:
> > > > access\_key: XXXXXX
> > > > secret\_key: XXXXXXXXX
> > > > discovery:
> > > > type: ec2
> > > > ec2:
> > > > groups: GROUP\_NAME
> > > >
> > > > cloud.node.auto\_attributes: true
> > > >
> > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > >
> > > > > Do you use the rpm/debia packages or is it an install from a tar ball/
> > > > > zip? Can you share the elasticsearch.yml configuration? Also, this can
> > > > > potentially be overridden using command line parameters (but I assume you
> > > > > don't do that).
> > > > >
> > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > >
> > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > >
> > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > > >
> > > > > > > Hi Daniel,
> > > > > > >
> > > > > > > It looks like you nodes do not discover each other - that number of
> > > > > > > nodes should be 3 for you.
> > > > > > >
> > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Boaz
> > > > > > >
> > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > >
> > > > > > > > so..
> > > > > > > > I decided to create a three nodes cluster (two data nodes and one
> > > > > > > > arbiter)
> > > > > > > > All three nodes are using ec2 discovery
> > > > > > > >
> > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I get
> > > > > > > > the following response:
> > > > > > > >
> > > > > > > > {
> > > > > > > >
> > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > - status: "red",
> > > > > > > > - timed\_out: false,
> > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > - active\_shards: 0,
> > > > > > > > - relocating\_shards: 0,
> > > > > > > > - initializing\_shards: 0,
> > > > > > > > - unassigned\_shards: 48
> > > > > > > >
> > > > > > > > }
> > > > > > > >
> > > > > > > > I can detect two troubling parameters:
> > > > > > > >
> > > > > > > > - active\_primary\_shards: 0
> > > > > > > > - AND
> > > > > > > > - unassigned\_shards: 48
> > > > > > > > -
> > > > > > > >
> > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > arbiter responding
> > > > > > > >
> > > > > > > > - status: "green"
> > > > > > > >
> > > > > > > > - witch makes me belive that they are not really connected to
> > > > > > > > the same cluster..
> > > > > > > >
> > > > > > > > -
> > > > > > > >
> > > > > > > > -
> > > > > > > > -
> > > > > > > >
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to a topic in
> > > > > > > > the Google Groups "elasticsearch" group.
> > > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > > > topic/elasticsearch/6cYLWYw6XHg/unsubscribe.
> > > > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google Groups
> > > > > "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send an
> > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 25, 2013, 10:59pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/10 "2013-11-25T22:59:19Z")
Then that's the issue.
I don't have experience with EC2 discovery, but you might want to try
unicast discovery to rule that out as the issue -
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)
web: [www.campaignmonitor.com](http://www.campaignmonitor.com)
On 26 November 2013 09:57, DanielR [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com) wrote:
> Nope...
> Both the arbiter and the second node are flagged with "new\_master"
>
> [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> new\_master [Node3][\][inet[/:9300]], reason: zen-disco-join
> (elected\_as\_master)
>
> [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> new\_master [Arbiter][\][inet[/:9300]], reason: zen-disco-join
> (elected\_as\_master)
>
> On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
>
> > 9300 is the port ES uses for intercluster comms, so that's fine.
> >
> > Browse through the logs on each of your nodes and see if you can find
> > references to the others joining the master.
> >
> > Regards,
> > Mark Walkom
> >
> > Infrastructure Engineer
> > Campaign Monitor
> > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> >
> > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> >
> > > ok 🙂
> > > found the logs
> > > I think maybe cluster.service is trying to find the other nodes on port
> > > 9300...
> > >
> > > so.. here it is:
> > >
> > > [2013-11-25 21:56:08,072][INFO][node]
> > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > [SCNode1Master] loaded [cloud-aws], sites
> > > [2013-11-25 21:56:13,667][INFO][node]
> > > [SCNode1Master] {0.90.1}[926]: initialized
> > > [2013-11-25 21:56:13,667][INFO][node]
> > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > [2013-11-25 21:56:13,849][INFO][transport]
> > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > publish\_address {inet[/:9300]}
> > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > reason: zen-disco-join (elected\_as\_master)
> > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > [SCNode1Master] SCluster/\
> > > [2013-11-25 21:56:17,053][INFO][http]
> > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > publish\_address {inet[/:9200]}
> > > [2013-11-25 21:56:17,053][INFO][node]
> > > [SCNode1Master] {0.90.1}[926]: started
> > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > ~
> > >
> > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > >
> > > > Hi Daniel,
> > > >
> > > > Just double checking - you do have space on the logs:
> > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > into.
> > > >
> > > > Cheers,
> > > > Boaz
> > > >
> > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > >
> > > > > I installed from tar.
> > > > >
> > > > > my configuration.yml looks like this:
> > > > >
> > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > SCluster
> > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > "SCNode1Master"
> > > > >
> > > > > node.master: true
> > > > > index.number\_of\_shards: 8
> > > > >
> > > > > index.number\_of\_replicas: 2
> > > > > path:
> > > > > logs: /var/log/elasticsearch
> > > > >
> > > > > cloud:
> > > > > aws:
> > > > > access\_key: XXXXXX
> > > > > secret\_key: XXXXXXXXX
> > > > > discovery:
> > > > > type: ec2
> > > > > ec2:
> > > > > groups: GROUP\_NAME
> > > > >
> > > > > cloud.node.auto\_attributes: true
> > > > >
> > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > >
> > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > you don't do that).
> > > > > >
> > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > >
> > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > >
> > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > > > >
> > > > > > > > Hi Daniel,
> > > > > > > >
> > > > > > > > It looks like you nodes do not discover each other - that number of
> > > > > > > > nodes should be 3 for you.
> > > > > > > >
> > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > > Boaz
> > > > > > > >
> > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > >
> > > > > > > > > so..
> > > > > > > > > I decided to create a three nodes cluster (two data nodes and one
> > > > > > > > > arbiter)
> > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > >
> > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I get
> > > > > > > > > the following response:
> > > > > > > > >
> > > > > > > > > {
> > > > > > > > >
> > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > - status: "red",
> > > > > > > > > - timed\_out: false,
> > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > - active\_shards: 0,
> > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > - unassigned\_shards: 48
> > > > > > > > >
> > > > > > > > > }
> > > > > > > > >
> > > > > > > > > I can detect two troubling parameters:
> > > > > > > > >
> > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > - AND
> > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > -
> > > > > > > > >
> > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > arbiter responding
> > > > > > > > >
> > > > > > > > > - status: "green"
> > > > > > > > >
> > > > > > > > > - witch makes me belive that they are not really connected to
> > > > > > > > > the same cluster..
> > > > > > > > >
> > > > > > > > > -
> > > > > > > > >
> > > > > > > > > -
> > > > > > > > > -
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > You received this message because you are subscribed to a topic in
> > > > > > > > > the Google Groups "elasticsearch" group.
> > > > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/to](https://groups.google.com/d/to)
> > > > > > > > > pic/elasticsearch/6cYLWYw6XHg/unsubscribe.
> > > > > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 25, 2013, 11:22pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/11 "2013-11-25T23:22:37Z")
ok.
so i used unicast and let's say i can live with this..
my cluster is acting all weird again...
still a red status and unassigned\_shards is 48
-
{
- cluster\_name: "SCluster",
- status: "red",
- timed\_out: false,
- number\_of\_nodes: 3,
- number\_of\_data\_nodes: 3,
- active\_primary\_shards: 8,
- active\_shards: 24,
- relocating\_shards: 0,
- initializing\_shards: 0,
- unassigned\_shards: 48
}
in my log i can see an entry
"dangling index, exists on local file system, but not in cluster metadata,
scheduling to delete in [2h], auto import to cluster state [YES]"
and i can't create it on the next node because "
IndexAlreadyExistsException"...
so... does the red status means??
On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> Then that's the issue.
>
> I don't have experience with EC2 discovery, but you might want to try
> unicast discovery to rule that out as the issue -
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html)[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \
> web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
>
> On 26 November 2013 09:57, DanielR \<[danielr...@gmail.com](mailto:danielr...@gmail.com) \\>wrote:
>
> > Nope...
> > Both the arbiter and the second node are flagged with "new\_master"
> >
> > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > new\_master [Node3][\][inet[/:9300]], reason: zen-disco-join
> > (elected\_as\_master)
> >
> > [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> > new\_master [Arbiter][\][inet[/:9300]], reason: zen-disco-join
> > (elected\_as\_master)
> >
> > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> >
> > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > >
> > > Browse through the logs on each of your nodes and see if you can find
> > > references to the others joining the master.
> > >
> > > Regards,
> > > Mark Walkom
> > >
> > > Infrastructure Engineer
> > > Campaign Monitor
> > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > >
> > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > >
> > > > ok 🙂
> > > > found the logs
> > > > I think maybe cluster.service is trying to find the other nodes on port
> > > > 9300...
> > > >
> > > > so.. here it is:
> > > >
> > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > publish\_address {inet[/:9300]}
> > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > reason: zen-disco-join (elected\_as\_master)
> > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > [SCNode1Master] SCluster/\
> > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > publish\_address {inet[/:9200]}
> > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > [SCNode1Master] {0.90.1}[926]: started
> > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > ~
> > > >
> > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > >
> > > > > Hi Daniel,
> > > > >
> > > > > Just double checking - you do have space on the logs:
> > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > into.
> > > > >
> > > > > Cheers,
> > > > > Boaz
> > > > >
> > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > >
> > > > > > I installed from tar.
> > > > > >
> > > > > > my configuration.yml looks like this:
> > > > > >
> > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > SCluster
> > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > "SCNode1Master"
> > > > > >
> > > > > > node.master: true
> > > > > > index.number\_of\_shards: 8
> > > > > >
> > > > > > index.number\_of\_replicas: 2
> > > > > > path:
> > > > > > logs: /var/log/elasticsearch
> > > > > >
> > > > > > cloud:
> > > > > > aws:
> > > > > > access\_key: XXXXXX
> > > > > > secret\_key: XXXXXXXXX
> > > > > > discovery:
> > > > > > type: ec2
> > > > > > ec2:
> > > > > > groups: GROUP\_NAME
> > > > > >
> > > > > > cloud.node.auto\_attributes: true
> > > > > >
> > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > >
> > > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > > you don't do that).
> > > > > > >
> > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > >
> > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > >
> > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > > > > >
> > > > > > > > > Hi Daniel,
> > > > > > > > >
> > > > > > > > > It looks like you nodes do not discover each other - that number
> > > > > > > > > of nodes should be 3 for you.
> > > > > > > > >
> > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Boaz
> > > > > > > > >
> > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > >
> > > > > > > > > > so..
> > > > > > > > > > I decided to create a three nodes cluster (two data nodes and one
> > > > > > > > > > arbiter)
> > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > >
> > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I get
> > > > > > > > > > the following response:
> > > > > > > > > >
> > > > > > > > > > {
> > > > > > > > > >
> > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > - status: "red",
> > > > > > > > > > - timed\_out: false,
> > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > >
> > > > > > > > > > }
> > > > > > > > > >
> > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > >
> > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > - AND
> > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > -
> > > > > > > > > >
> > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > arbiter responding
> > > > > > > > > >
> > > > > > > > > > - status: "green"
> > > > > > > > > >
> > > > > > > > > > - witch makes me belive that they are not really connected to
> > > > > > > > > > the same cluster..
> > > > > > > > > >
> > > > > > > > > > -
> > > > > > > > > >
> > > > > > > > > > -
> > > > > > > > > > -
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > You received this message because you are subscribed to a topic in
> > > > > > > > > > the Google Groups "elasticsearch" group.
> > > > > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > > > > > topic/elasticsearch/6cYLWYw6XHg/unsubscribe.
> > > > > > > > > > To unsubscribe from this group and all its topics, send an email to
> > > > > > > > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > > >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to the Google
> > > > > > > Groups "elasticsearch" group.
> > > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [November 25, 2013, 11:27pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/12 "2013-11-25T23:27:46Z")
It's red because you will have unassigned primaries.
Given your previous issues you might want to reload the data into new
indexes and start afresh?
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com)
web: [www.campaignmonitor.com](http://www.campaignmonitor.com)
On 26 November 2013 10:22, DanielR [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com) wrote:
> ok.
> so i used unicast and let's say i can live with this..
>
> my cluster is acting all weird again...
>
> still a red status and unassigned\_shards is 48
>
> -
>
> {
>
> - cluster\_name: "SCluster",
> - status: "red",
> - timed\_out: false,
> - number\_of\_nodes: 3,
> - number\_of\_data\_nodes: 3,
> - active\_primary\_shards: 8,
> - active\_shards: 24,
> - relocating\_shards: 0,
> - initializing\_shards: 0,
> - unassigned\_shards: 48
>
> }
>
> in my log i can see an entry
> "dangling index, exists on local file system, but not in cluster
> metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
>
> and i can't create it on the next node because "
> IndexAlreadyExistsException"...
>
> so... does the red status means??
>
> On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
>
> > Then that's the issue.
> >
> > I don't have experience with EC2 discovery, but you might want to try
> > unicast discovery to rule that out as the issue -
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> >
> > Regards,
> > Mark Walkom
> >
> > Infrastructure Engineer
> > Campaign Monitor
> > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> >
> > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> >
> > > Nope...
> > > Both the arbiter and the second node are flagged with "new\_master"
> > >
> > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > new\_master [Node3][\][inet[/:9300]], reason:
> > > zen-disco-join (elected\_as\_master)
> > >
> > > [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> > > new\_master [Arbiter][\][inet[/:9300]], reason:
> > > zen-disco-join (elected\_as\_master)
> > >
> > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > >
> > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > >
> > > > Browse through the logs on each of your nodes and see if you can find
> > > > references to the others joining the master.
> > > >
> > > > Regards,
> > > > Mark Walkom
> > > >
> > > > Infrastructure Engineer
> > > > Campaign Monitor
> > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > >
> > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > >
> > > > > ok 🙂
> > > > > found the logs
> > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > port 9300...
> > > > >
> > > > > so.. here it is:
> > > > >
> > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > publish\_address {inet[/:9300]}
> > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > [SCNode1Master] SCluster/\
> > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > publish\_address {inet[/:9200]}
> > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > ~
> > > > >
> > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > >
> > > > > > Hi Daniel,
> > > > > >
> > > > > > Just double checking - you do have space on the logs:
> > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > into.
> > > > > >
> > > > > > Cheers,
> > > > > > Boaz
> > > > > >
> > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > >
> > > > > > > I installed from tar.
> > > > > > >
> > > > > > > my configuration.yml looks like this:
> > > > > > >
> > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > SCluster
> > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > "SCNode1Master"
> > > > > > >
> > > > > > > node.master: true
> > > > > > > index.number\_of\_shards: 8
> > > > > > >
> > > > > > > index.number\_of\_replicas: 2
> > > > > > > path:
> > > > > > > logs: /var/log/elasticsearch
> > > > > > >
> > > > > > > cloud:
> > > > > > > aws:
> > > > > > > access\_key: XXXXXX
> > > > > > > secret\_key: XXXXXXXXX
> > > > > > > discovery:
> > > > > > > type: ec2
> > > > > > > ec2:
> > > > > > > groups: GROUP\_NAME
> > > > > > >
> > > > > > > cloud.node.auto\_attributes: true
> > > > > > >
> > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > >
> > > > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > > > you don't do that).
> > > > > > > >
> > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > >
> > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > >
> > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > >
> > > > > > > > > > Hi Daniel,
> > > > > > > > > >
> > > > > > > > > > It looks like you nodes do not discover each other - that number
> > > > > > > > > > of nodes should be 3 for you.
> > > > > > > > > >
> > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > > Boaz
> > > > > > > > > >
> > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > >
> > > > > > > > > > > so..
> > > > > > > > > > > I decided to create a three nodes cluster (two data nodes and
> > > > > > > > > > > one arbiter)
> > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > >
> > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I
> > > > > > > > > > > get the following response:
> > > > > > > > > > >
> > > > > > > > > > > {
> > > > > > > > > > >
> > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > - status: "red",
> > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > >
> > > > > > > > > > > }
> > > > > > > > > > >
> > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > >
> > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > - AND
> > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > -
> > > > > > > > > > >
> > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > arbiter responding
> > > > > > > > > > >
> > > > > > > > > > > - status: "green"
> > > > > > > > > > >
> > > > > > > > > > > - witch makes me belive that they are not really connected
> > > > > > > > > > > to the same cluster..
> > > > > > > > > > >
> > > > > > > > > > > -
> > > > > > > > > > >
> > > > > > > > > > > -
> > > > > > > > > > > -
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > You received this message because you are subscribed to a topic in
> > > > > > > > > > > the Google Groups "elasticsearch" group.
> > > > > > > > > > > To unsubscribe from this topic, visit [https://groups.google.com/d/](https://groups.google.com/d/)
> > > > > > > > > > > topic/elasticsearch/6cYLWYw6XHg/unsubscribe.
> > > > > > > > > > > To unsubscribe from this group and all its topics, send an email
> > > > > > > > > > > to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > > > >
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > Groups "elasticsearch" group.
> > > > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 25, 2013, 11:30pm UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/13 "2013-11-25T23:30:30Z")
I've deleted all of my data and now i have "only" 24 unassigned\_shards..
On Tuesday, November 26, 2013 1:27:46 AM UTC+2, Mark Walkom wrote:
> It's red because you will have unassigned primaries.
>
> Given your previous issues you might want to reload the data into new
> indexes and start afresh?
>
> Regards,
> Mark Walkom
>
> Infrastructure Engineer
> Campaign Monitor
> email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \
> web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
>
> On 26 November 2013 10:22, DanielR \<[danielr...@gmail.com](mailto:danielr...@gmail.com) \\>wrote:
>
> > ok.
> > so i used unicast and let's say i can live with this..
> >
> > my cluster is acting all weird again...
> >
> > still a red status and unassigned\_shards is 48
> >
> > -
> >
> > {
> >
> > - cluster\_name: "SCluster",
> > - status: "red",
> > - timed\_out: false,
> > - number\_of\_nodes: 3,
> > - number\_of\_data\_nodes: 3,
> > - active\_primary\_shards: 8,
> > - active\_shards: 24,
> > - relocating\_shards: 0,
> > - initializing\_shards: 0,
> > - unassigned\_shards: 48
> >
> > }
> >
> > in my log i can see an entry
> > "dangling index, exists on local file system, but not in cluster
> > metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
> >
> > and i can't create it on the next node because "
> > IndexAlreadyExistsException"...
> >
> > so... does the red status means??
> >
> > On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> >
> > > Then that's the issue.
> > >
> > > I don't have experience with EC2 discovery, but you might want to try
> > > unicast discovery to rule that out as the issue -
> > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> > >
> > > Regards,
> > > Mark Walkom
> > >
> > > Infrastructure Engineer
> > > Campaign Monitor
> > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > >
> > > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > >
> > > > Nope...
> > > > Both the arbiter and the second node are flagged with "new\_master"
> > > >
> > > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > > new\_master [Node3][\][inet[/:9300]], reason:
> > > > zen-disco-join (elected\_as\_master)
> > > >
> > > > [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> > > > new\_master [Arbiter][\][inet[/:9300]], reason:
> > > > zen-disco-join (elected\_as\_master)
> > > >
> > > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > > >
> > > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > > >
> > > > > Browse through the logs on each of your nodes and see if you can find
> > > > > references to the others joining the master.
> > > > >
> > > > > Regards,
> > > > > Mark Walkom
> > > > >
> > > > > Infrastructure Engineer
> > > > > Campaign Monitor
> > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > >
> > > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > >
> > > > > > ok 🙂
> > > > > > found the logs
> > > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > > port 9300...
> > > > > >
> > > > > > so.. here it is:
> > > > > >
> > > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > > publish\_address {inet[/:9300]}
> > > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > > [SCNode1Master] SCluster/\
> > > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > > publish\_address {inet[/:9200]}
> > > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > > ~
> > > > > >
> > > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > > >
> > > > > > > Hi Daniel,
> > > > > > >
> > > > > > > Just double checking - you do have space on the logs:
> > > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > > into.
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Boaz
> > > > > > >
> > > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > > >
> > > > > > > > I installed from tar.
> > > > > > > >
> > > > > > > > my configuration.yml looks like this:
> > > > > > > >
> > > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > SCluster
> > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > "SCNode1Master"
> > > > > > > >
> > > > > > > > node.master: true
> > > > > > > > index.number\_of\_shards: 8
> > > > > > > >
> > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > path:
> > > > > > > > logs: /var/log/elasticsearch
> > > > > > > >
> > > > > > > > cloud:
> > > > > > > > aws:
> > > > > > > > access\_key: XXXXXX
> > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > discovery:
> > > > > > > > type: ec2
> > > > > > > > ec2:
> > > > > > > > groups: GROUP\_NAME
> > > > > > > >
> > > > > > > > cloud.node.auto\_attributes: true
> > > > > > > >
> > > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > > >
> > > > > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > > > > you don't do that).
> > > > > > > > >
> > > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > > >
> > > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > > >
> > > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Daniel,
> > > > > > > > > > >
> > > > > > > > > > > It looks like you nodes do not discover each other - that number
> > > > > > > > > > > of nodes should be 3 for you.
> > > > > > > > > > >
> > > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > > >
> > > > > > > > > > > Cheers,
> > > > > > > > > > > Boaz
> > > > > > > > > > >
> > > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > > >
> > > > > > > > > > > > so..
> > > > > > > > > > > > I decided to create a three nodes cluster (two data nodes and
> > > > > > > > > > > > one arbiter)
> > > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > > >
> > > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I
> > > > > > > > > > > > get the following response:
> > > > > > > > > > > >
> > > > > > > > > > > > {
> > > > > > > > > > > >
> > > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > > - status: "red",
> > > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > >
> > > > > > > > > > > > }
> > > > > > > > > > > >
> > > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > > >
> > > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > > - AND
> > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > -
> > > > > > > > > > > >
> > > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > > arbiter responding
> > > > > > > > > > > >
> > > > > > > > > > > > - status: "green"
> > > > > > > > > > > >
> > > > > > > > > > > > - witch makes me belive that they are not really connected
> > > > > > > > > > > > to the same cluster..
> > > > > > > > > > > >
> > > > > > > > > > > > -
> > > > > > > > > > > >
> > > > > > > > > > > > -
> > > > > > > > > > > > -
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > You received this message because you are subscribed to a topic
> > > > > > > > > > > > in the Google Groups "elasticsearch" group.
> > > > > > > > > > > > To unsubscribe from this topic, visit
> > > > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/)
> > > > > > > > > > > > unsubscribe.
> > > > > > > > > > > > To unsubscribe from this group and all its topics, send an email
> > > > > > > > > > > > to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > > Groups "elasticsearch" group.
> > > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > >
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send an
> > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [November 26, 2013, 6:25am UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/14 "2013-11-26T06:25:56Z")
Hi Daniel,
The nodes need to be able to communicate with each other on port 9300 . Did
you set up a security group for the servers that allows access on port 9300
between them?
I think you have a configuration issue. You don't need to use unicast
discovery on ec2 but rather the discovery mechanism supplied by the aws
plugin. Let's first find the cause of that and then look at the unassigned
shards as they will probably be caused by the same thing.
Where did you find the logs in the end?
Cheers,
Boaz
On Tuesday, November 26, 2013 12:30:30 AM UTC+1, DanielR wrote:
> I've deleted all of my data and now i have "only" 24 unassigned\_shards..
>
> On Tuesday, November 26, 2013 1:27:46 AM UTC+2, Mark Walkom wrote:
>
> > It's red because you will have unassigned primaries.
> >
> > Given your previous issues you might want to reload the data into new
> > indexes and start afresh?
> >
> > Regards,
> > Mark Walkom
> >
> > Infrastructure Engineer
> > Campaign Monitor
> > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> >
> > On 26 November 2013 10:22, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> >
> > > ok.
> > > so i used unicast and let's say i can live with this..
> > >
> > > my cluster is acting all weird again...
> > >
> > > still a red status and unassigned\_shards is 48
> > >
> > > -
> > >
> > > {
> > >
> > > - cluster\_name: "SCluster",
> > > - status: "red",
> > > - timed\_out: false,
> > > - number\_of\_nodes: 3,
> > > - number\_of\_data\_nodes: 3,
> > > - active\_primary\_shards: 8,
> > > - active\_shards: 24,
> > > - relocating\_shards: 0,
> > > - initializing\_shards: 0,
> > > - unassigned\_shards: 48
> > >
> > > }
> > >
> > > in my log i can see an entry
> > > "dangling index, exists on local file system, but not in cluster
> > > metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
> > >
> > > and i can't create it on the next node because "
> > > IndexAlreadyExistsException"...
> > >
> > > so... does the red status means??
> > >
> > > On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> > >
> > > > Then that's the issue.
> > > >
> > > > I don't have experience with EC2 discovery, but you might want to try
> > > > unicast discovery to rule that out as the issue -
> > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > > > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> > > >
> > > > Regards,
> > > > Mark Walkom
> > > >
> > > > Infrastructure Engineer
> > > > Campaign Monitor
> > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > >
> > > > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > >
> > > > > Nope...
> > > > > Both the arbiter and the second node are flagged with "new\_master"
> > > > >
> > > > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > > > new\_master [Node3][\][inet[/:9300]], reason:
> > > > > zen-disco-join (elected\_as\_master)
> > > > >
> > > > > [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> > > > > new\_master [Arbiter][\][inet[/:9300]], reason:
> > > > > zen-disco-join (elected\_as\_master)
> > > > >
> > > > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > > > >
> > > > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > > > >
> > > > > > Browse through the logs on each of your nodes and see if you can find
> > > > > > references to the others joining the master.
> > > > > >
> > > > > > Regards,
> > > > > > Mark Walkom
> > > > > >
> > > > > > Infrastructure Engineer
> > > > > > Campaign Monitor
> > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > >
> > > > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > >
> > > > > > > ok 🙂
> > > > > > > found the logs
> > > > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > > > port 9300...
> > > > > > >
> > > > > > > so.. here it is:
> > > > > > >
> > > > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > > > publish\_address {inet[/:9300]}
> > > > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > > > [SCNode1Master] SCluster/\
> > > > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > > > publish\_address {inet[/:9200]}
> > > > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > > > ~
> > > > > > >
> > > > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > > > >
> > > > > > > > Hi Daniel,
> > > > > > > >
> > > > > > > > Just double checking - you do have space on the logs:
> > > > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > > > into.
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > > Boaz
> > > > > > > >
> > > > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > > > >
> > > > > > > > > I installed from tar.
> > > > > > > > >
> > > > > > > > > my configuration.yml looks like this:
> > > > > > > > >
> > > > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > > SCluster
> > > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > > "SCNode1Master"
> > > > > > > > >
> > > > > > > > > node.master: true
> > > > > > > > > index.number\_of\_shards: 8
> > > > > > > > >
> > > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > > path:
> > > > > > > > > logs: /var/log/elasticsearch
> > > > > > > > >
> > > > > > > > > cloud:
> > > > > > > > > aws:
> > > > > > > > > access\_key: XXXXXX
> > > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > > discovery:
> > > > > > > > > type: ec2
> > > > > > > > > ec2:
> > > > > > > > > groups: GROUP\_NAME
> > > > > > > > >
> > > > > > > > > cloud.node.auto\_attributes: true
> > > > > > > > >
> > > > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > >
> > > > > > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > > > > > you don't do that).
> > > > > > > > > >
> > > > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > > > >
> > > > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > > > >
> > > > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Daniel,
> > > > > > > > > > > >
> > > > > > > > > > > > It looks like you nodes do not discover each other - that
> > > > > > > > > > > > number of nodes should be 3 for you.
> > > > > > > > > > > >
> > > > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers,
> > > > > > > > > > > > Boaz
> > > > > > > > > > > >
> > > > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > so..
> > > > > > > > > > > > > I decided to create a three nodes cluster (two data nodes and
> > > > > > > > > > > > > one arbiter)
> > > > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > > > >
> > > > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I
> > > > > > > > > > > > > get the following response:
> > > > > > > > > > > > >
> > > > > > > > > > > > > {
> > > > > > > > > > > > >
> > > > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > > > - status: "red",
> > > > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > >
> > > > > > > > > > > > > }
> > > > > > > > > > > > >
> > > > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > > > >
> > > > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > > > - AND
> > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > -
> > > > > > > > > > > > >
> > > > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > > > arbiter responding
> > > > > > > > > > > > >
> > > > > > > > > > > > > - status: "green"
> > > > > > > > > > > > >
> > > > > > > > > > > > > - witch makes me belive that they are not really connected
> > > > > > > > > > > > > to the same cluster..
> > > > > > > > > > > > >
> > > > > > > > > > > > > -
> > > > > > > > > > > > >
> > > > > > > > > > > > > -
> > > > > > > > > > > > > -
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > You received this message because you are subscribed to a topic
> > > > > > > > > > > > > in the Google Groups "elasticsearch" group.
> > > > > > > > > > > > > To unsubscribe from this topic, visit
> > > > > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/)
> > > > > > > > > > > > > unsubscribe.
> > > > > > > > > > > > > To unsubscribe from this group and all its topics, send an email
> > > > > > > > > > > > > to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out)
> > > > > > > > > > > > > .
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > > > Groups "elasticsearch" group.
> > > > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 26, 2013, 6:29am UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/15 "2013-11-26T06:29:11Z")
The logs were found on the es\_home directory, like you suggested.
And i did opened the ports (9200-9300) on the security group..
Unicast is using the same port, no?
Sent from my iPhone.
דניאל
0546350772
[http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
On 26 בנוב 2013, at 08:26, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
Hi Daniel,
The nodes need to be able to communicate with each other on port 9300 . Did
you set up a security group for the servers that allows access on port 9300
between them?
I think you have a configuration issue. You don't need to use unicast
discovery on ec2 but rather the discovery mechanism supplied by the aws
plugin. Let's first find the cause of that and then look at the unassigned
shards as they will probably be caused by the same thing.
Where did you find the logs in the end?
Cheers,
Boaz
On Tuesday, November 26, 2013 12:30:30 AM UTC+1, DanielR wrote:
> I've deleted all of my data and now i have "only" 24 unassigned\_shards..
>
> On Tuesday, November 26, 2013 1:27:46 AM UTC+2, Mark Walkom wrote:
>
> > It's red because you will have unassigned primaries.
> >
> > Given your previous issues you might want to reload the data into new
> > indexes and start afresh?
> >
> > Regards,
> > Mark Walkom
> >
> > Infrastructure Engineer
> > Campaign Monitor
> > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> >
> > On 26 November 2013 10:22, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> >
> > > ok.
> > > so i used unicast and let's say i can live with this..
> > >
> > > my cluster is acting all weird again...
> > >
> > > still a red status and unassigned\_shards is 48
> > >
> > > -
> > >
> > > {
> > >
> > > - cluster\_name: "SCluster",
> > > - status: "red",
> > > - timed\_out: false,
> > > - number\_of\_nodes: 3,
> > > - number\_of\_data\_nodes: 3,
> > > - active\_primary\_shards: 8,
> > > - active\_shards: 24,
> > > - relocating\_shards: 0,
> > > - initializing\_shards: 0,
> > > - unassigned\_shards: 48
> > >
> > > }
> > >
> > > in my log i can see an entry
> > > "dangling index, exists on local file system, but not in cluster
> > > metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
> > >
> > > and i can't create it on the next node because "
> > > IndexAlreadyExistsException"...
> > >
> > > so... does the red status means??
> > >
> > > On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> > >
> > > > Then that's the issue.
> > > >
> > > > I don't have experience with EC2 discovery, but you might want to try
> > > > unicast discovery to rule that out as the issue -
> > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > > > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> > > >
> > > > Regards,
> > > > Mark Walkom
> > > >
> > > > Infrastructure Engineer
> > > > Campaign Monitor
> > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > >
> > > > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > >
> > > > > Nope...
> > > > > Both the arbiter and the second node are flagged with "new\_master"
> > > > >
> > > > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > > > new\_master [Node3][\][inet[/:9300]], reason:
> > > > > zen-disco-join (elected\_as\_master)
> > > > >
> > > > > [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> > > > > new\_master [Arbiter][\][inet[/:9300]], reason:
> > > > > zen-disco-join (elected\_as\_master)
> > > > >
> > > > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > > > >
> > > > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > > > >
> > > > > > Browse through the logs on each of your nodes and see if you can find
> > > > > > references to the others joining the master.
> > > > > >
> > > > > > Regards,
> > > > > > Mark Walkom
> > > > > >
> > > > > > Infrastructure Engineer
> > > > > > Campaign Monitor
> > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > >
> > > > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > >
> > > > > > > ok 🙂
> > > > > > > found the logs
> > > > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > > > port 9300...
> > > > > > >
> > > > > > > so.. here it is:
> > > > > > >
> > > > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > > > publish\_address {inet[/:9300]}
> > > > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > > > [SCNode1Master] SCluster/\
> > > > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > > > publish\_address {inet[/:9200]}
> > > > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > > > ~
> > > > > > >
> > > > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > > > >
> > > > > > > > Hi Daniel,
> > > > > > > >
> > > > > > > > Just double checking - you do have space on the logs:
> > > > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > > > into.
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > > Boaz
> > > > > > > >
> > > > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > > > >
> > > > > > > > > I installed from tar.
> > > > > > > > >
> > > > > > > > > my configuration.yml looks like this:
> > > > > > > > >
> > > > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > > SCluster
> > > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > > "SCNode1Master"
> > > > > > > > >
> > > > > > > > > node.master: true
> > > > > > > > > index.number\_of\_shards: 8
> > > > > > > > >
> > > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > > path:
> > > > > > > > > logs: /var/log/elasticsearch
> > > > > > > > >
> > > > > > > > > cloud:
> > > > > > > > > aws:
> > > > > > > > > access\_key: XXXXXX
> > > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > > discovery:
> > > > > > > > > type: ec2
> > > > > > > > > ec2:
> > > > > > > > > groups: GROUP\_NAME
> > > > > > > > >
> > > > > > > > > cloud.node.auto\_attributes: true
> > > > > > > > >
> > > > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > >
> > > > > > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > > > > > you don't do that).
> > > > > > > > > >
> > > > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > > > >
> > > > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > > > >
> > > > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Daniel,
> > > > > > > > > > > >
> > > > > > > > > > > > It looks like you nodes do not discover each other - that
> > > > > > > > > > > > number of nodes should be 3 for you.
> > > > > > > > > > > >
> > > > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers,
> > > > > > > > > > > > Boaz
> > > > > > > > > > > >
> > > > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > so..
> > > > > > > > > > > > > I decided to create a three nodes cluster (two data nodes and
> > > > > > > > > > > > > one arbiter)
> > > > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > > > >
> > > > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I
> > > > > > > > > > > > > get the following response:
> > > > > > > > > > > > >
> > > > > > > > > > > > > {
> > > > > > > > > > > > >
> > > > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > > > - status: "red",
> > > > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > >
> > > > > > > > > > > > > }
> > > > > > > > > > > > >
> > > > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > > > >
> > > > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > > > - AND
> > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > -
> > > > > > > > > > > > >
> > > > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > > > arbiter responding
> > > > > > > > > > > > >
> > > > > > > > > > > > > - status: "green"
> > > > > > > > > > > > >
> > > > > > > > > > > > > - witch makes me belive that they are not really connected
> > > > > > > > > > > > > to the same cluster..
> > > > > > > > > > > > >
> > > > > > > > > > > > > -
> > > > > > > > > > > > >
> > > > > > > > > > > > > -
> > > > > > > > > > > > > -
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > You received this message because you are subscribed to a topic
> > > > > > > > > > > > > in the Google Groups "elasticsearch" group.
> > > > > > > > > > > > > To unsubscribe from this topic, visit
> > > > > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/)
> > > > > > > > > > > > > unsubscribe.
> > > > > > > > > > > > > To unsubscribe from this group and all its topics, send an email
> > > > > > > > > > > > > to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out)
> > > > > > > > > > > > > .
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > > > Groups "elasticsearch" group.
> > > > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "elasticsearch" group.
> > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [November 26, 2013, 8:27am UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/16 "2013-11-26T08:27:59Z")
Hi Daniel,
ES will use 9300 for internal cross node communication (actually 9300-9400
but that's only relevant if you run multiple nodes on a single server).
9200 is used for HTTP request to the API.
Given the fact that the logging settings were not picked up I think the
source of your problem is the settings file. I'd check it for correct
indentation / structure.
Cheers,
Boaz
On Tue, Nov 26, 2013 at 7:29 AM, Daniel Rasta [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com)wrote:
> The logs were found on the es\_home directory, like you suggested.
> And i did opened the ports (9200-9300) on the security group..
>
> Unicast is using the same port, no?
>
> Sent from my iPhone.
> דניאל
> 0546350772
> [http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
>
> On 26 בנוב 2013, at 08:26, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
>
> Hi Daniel,
>
> The nodes need to be able to communicate with each other on port 9300 .
> Did you set up a security group for the servers that allows access on port
> 9300 between them?
>
> I think you have a configuration issue. You don't need to use unicast
> discovery on ec2 but rather the discovery mechanism supplied by the aws
> plugin. Let's first find the cause of that and then look at the unassigned
> shards as they will probably be caused by the same thing.
>
> Where did you find the logs in the end?
>
> Cheers,
> Boaz
>
> On Tuesday, November 26, 2013 12:30:30 AM UTC+1, DanielR wrote:
>
> > I've deleted all of my data and now i have "only" 24 unassigned\_shards..
> >
> > On Tuesday, November 26, 2013 1:27:46 AM UTC+2, Mark Walkom wrote:
> >
> > > It's red because you will have unassigned primaries.
> > >
> > > Given your previous issues you might want to reload the data into new
> > > indexes and start afresh?
> > >
> > > Regards,
> > > Mark Walkom
> > >
> > > Infrastructure Engineer
> > > Campaign Monitor
> > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > >
> > > On 26 November 2013 10:22, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > >
> > > > ok.
> > > > so i used unicast and let's say i can live with this..
> > > >
> > > > my cluster is acting all weird again...
> > > >
> > > > still a red status and unassigned\_shards is 48
> > > >
> > > > -
> > > >
> > > > {
> > > >
> > > > - cluster\_name: "SCluster",
> > > > - status: "red",
> > > > - timed\_out: false,
> > > > - number\_of\_nodes: 3,
> > > > - number\_of\_data\_nodes: 3,
> > > > - active\_primary\_shards: 8,
> > > > - active\_shards: 24,
> > > > - relocating\_shards: 0,
> > > > - initializing\_shards: 0,
> > > > - unassigned\_shards: 48
> > > >
> > > > }
> > > >
> > > > in my log i can see an entry
> > > > "dangling index, exists on local file system, but not in cluster
> > > > metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
> > > >
> > > > and i can't create it on the next node because "
> > > > IndexAlreadyExistsException"...
> > > >
> > > > so... does the red status means??
> > > >
> > > > On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> > > >
> > > > > Then that's the issue.
> > > > >
> > > > > I don't have experience with EC2 discovery, but you might want to try
> > > > > unicast discovery to rule that out as the issue -
> > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > > > > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> > > > >
> > > > > Regards,
> > > > > Mark Walkom
> > > > >
> > > > > Infrastructure Engineer
> > > > > Campaign Monitor
> > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > >
> > > > > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > >
> > > > > > Nope...
> > > > > > Both the arbiter and the second node are flagged with "new\_master"
> > > > > >
> > > > > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > > > > new\_master [Node3][\][inet[/:9300]], reason:
> > > > > > zen-disco-join (elected\_as\_master)
> > > > > >
> > > > > > [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> > > > > > new\_master [Arbiter][\][inet[/:9300]], reason:
> > > > > > zen-disco-join (elected\_as\_master)
> > > > > >
> > > > > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > > > > >
> > > > > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > > > > >
> > > > > > > Browse through the logs on each of your nodes and see if you can
> > > > > > > find references to the others joining the master.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Mark Walkom
> > > > > > >
> > > > > > > Infrastructure Engineer
> > > > > > > Campaign Monitor
> > > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > > >
> > > > > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > > >
> > > > > > > > ok 🙂
> > > > > > > > found the logs
> > > > > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > > > > port 9300...
> > > > > > > >
> > > > > > > > so.. here it is:
> > > > > > > >
> > > > > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > > > > publish\_address {inet[/:9300]}
> > > > > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > > > > [SCNode1Master] SCluster/\
> > > > > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > > > > publish\_address {inet[/:9200]}
> > > > > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > > > > ~
> > > > > > > >
> > > > > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > > > > >
> > > > > > > > > Hi Daniel,
> > > > > > > > >
> > > > > > > > > Just double checking - you do have space on the logs:
> > > > > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > > > > into.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Boaz
> > > > > > > > >
> > > > > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > > > > >
> > > > > > > > > > I installed from tar.
> > > > > > > > > >
> > > > > > > > > > my configuration.yml looks like this:
> > > > > > > > > >
> > > > > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > > > SCluster
> > > > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > > > "SCNode1Master"
> > > > > > > > > >
> > > > > > > > > > node.master: true
> > > > > > > > > > index.number\_of\_shards: 8
> > > > > > > > > >
> > > > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > > > path:
> > > > > > > > > > logs: /var/log/elasticsearch
> > > > > > > > > >
> > > > > > > > > > cloud:
> > > > > > > > > > aws:
> > > > > > > > > > access\_key: XXXXXX
> > > > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > > > discovery:
> > > > > > > > > > type: ec2
> > > > > > > > > > ec2:
> > > > > > > > > > groups: GROUP\_NAME
> > > > > > > > > >
> > > > > > > > > > cloud.node.auto\_attributes: true
> > > > > > > > > >
> > > > > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > > >
> > > > > > > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > > > > > > you don't do that).
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > > > > >
> > > > > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > > > > >
> > > > > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi Daniel,
> > > > > > > > > > > > >
> > > > > > > > > > > > > It looks like you nodes do not discover each other - that
> > > > > > > > > > > > > number of nodes should be 3 for you.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > Boaz
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > so..
> > > > > > > > > > > > > > I decided to create a three nodes cluster (two data nodes and
> > > > > > > > > > > > > > one arbiter)
> > > > > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I
> > > > > > > > > > > > > > get the following response:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > {
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > > > > - status: "red",
> > > > > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > }
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > > > > - AND
> > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > > > > arbiter responding
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - status: "green"
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - witch makes me belive that they are not really
> > > > > > > > > > > > > > connected to the same cluster..
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > You received this message because you are subscribed to a topic
> > > > > > > > > > > > > > in the Google Groups "elasticsearch" group.
> > > > > > > > > > > > > > To unsubscribe from this topic, visit
> > > > > > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/)
> > > > > > > > > > > > > > unsubscribe.
> > > > > > > > > > > > > > To unsubscribe from this group and all its topics, send an
> > > > > > > > > > > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > > > For more options, visit [https://groups.google.com/grou](https://groups.google.com/grou)
> > > > > > > > > > > > > > ps/opt\_out.
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > > > > Groups "elasticsearch" group.
> > > > > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > > >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to the Google
> > > > > > > Groups "elasticsearch" group.
> > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 26, 2013, 10:33am UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/17 "2013-11-26T10:33:44Z")
Sounds great! But is there something I can use to verify my config file?
Also, every other setting entry seems to work just fine..
My main consideration is the shards allocation. Even with the unicast
discovery, they should be allocated.
Sent from my iPhone.
דניאל
0546350772
[http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
On 26 בנוב 2013, at 10:28, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
Hi Daniel,
ES will use 9300 for internal cross node communication (actually 9300-9400
but that's only relevant if you run multiple nodes on a single server).
9200 is used for HTTP request to the API.
Given the fact that the logging settings were not picked up I think the
source of your problem is the settings file. I'd check it for correct
indentation / structure.
Cheers,
Boaz
On Tue, Nov 26, 2013 at 7:29 AM, Daniel Rasta [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com)wrote:
> The logs were found on the es\_home directory, like you suggested.
> And i did opened the ports (9200-9300) on the security group..
>
> Unicast is using the same port, no?
>
> Sent from my iPhone.
> דניאל
> 0546350772
> [http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
>
> On 26 בנוב 2013, at 08:26, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
>
> Hi Daniel,
>
> The nodes need to be able to communicate with each other on port 9300 .
> Did you set up a security group for the servers that allows access on port
> 9300 between them?
>
> I think you have a configuration issue. You don't need to use unicast
> discovery on ec2 but rather the discovery mechanism supplied by the aws
> plugin. Let's first find the cause of that and then look at the unassigned
> shards as they will probably be caused by the same thing.
>
> Where did you find the logs in the end?
>
> Cheers,
> Boaz
>
> On Tuesday, November 26, 2013 12:30:30 AM UTC+1, DanielR wrote:
>
> > I've deleted all of my data and now i have "only" 24 unassigned\_shards..
> >
> > On Tuesday, November 26, 2013 1:27:46 AM UTC+2, Mark Walkom wrote:
> >
> > > It's red because you will have unassigned primaries.
> > >
> > > Given your previous issues you might want to reload the data into new
> > > indexes and start afresh?
> > >
> > > Regards,
> > > Mark Walkom
> > >
> > > Infrastructure Engineer
> > > Campaign Monitor
> > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > >
> > > On 26 November 2013 10:22, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > >
> > > > ok.
> > > > so i used unicast and let's say i can live with this..
> > > >
> > > > my cluster is acting all weird again...
> > > >
> > > > still a red status and unassigned\_shards is 48
> > > >
> > > > -
> > > >
> > > > {
> > > >
> > > > - cluster\_name: "SCluster",
> > > > - status: "red",
> > > > - timed\_out: false,
> > > > - number\_of\_nodes: 3,
> > > > - number\_of\_data\_nodes: 3,
> > > > - active\_primary\_shards: 8,
> > > > - active\_shards: 24,
> > > > - relocating\_shards: 0,
> > > > - initializing\_shards: 0,
> > > > - unassigned\_shards: 48
> > > >
> > > > }
> > > >
> > > > in my log i can see an entry
> > > > "dangling index, exists on local file system, but not in cluster
> > > > metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
> > > >
> > > > and i can't create it on the next node because "
> > > > IndexAlreadyExistsException"...
> > > >
> > > > so... does the red status means??
> > > >
> > > > On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> > > >
> > > > > Then that's the issue.
> > > > >
> > > > > I don't have experience with EC2 discovery, but you might want to try
> > > > > unicast discovery to rule that out as the issue -
> > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > > > > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> > > > >
> > > > > Regards,
> > > > > Mark Walkom
> > > > >
> > > > > Infrastructure Engineer
> > > > > Campaign Monitor
> > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > >
> > > > > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > >
> > > > > > Nope...
> > > > > > Both the arbiter and the second node are flagged with "new\_master"
> > > > > >
> > > > > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > > > > new\_master [Node3][\][inet[/:9300]], reason:
> > > > > > zen-disco-join (elected\_as\_master)
> > > > > >
> > > > > > [2013-11-25 22:49:44,079][INFO][cluster.service] [Arbiter]
> > > > > > new\_master [Arbiter][\][inet[/:9300]], reason:
> > > > > > zen-disco-join (elected\_as\_master)
> > > > > >
> > > > > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > > > > >
> > > > > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > > > > >
> > > > > > > Browse through the logs on each of your nodes and see if you can
> > > > > > > find references to the others joining the master.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Mark Walkom
> > > > > > >
> > > > > > > Infrastructure Engineer
> > > > > > > Campaign Monitor
> > > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > > >
> > > > > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > > >
> > > > > > > > ok 🙂
> > > > > > > > found the logs
> > > > > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > > > > port 9300...
> > > > > > > >
> > > > > > > > so.. here it is:
> > > > > > > >
> > > > > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > > > > publish\_address {inet[/:9300]}
> > > > > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > > > > [SCNode1Master] SCluster/\
> > > > > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > > > > publish\_address {inet[/:9200]}
> > > > > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > > > > ~
> > > > > > > >
> > > > > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > > > > >
> > > > > > > > > Hi Daniel,
> > > > > > > > >
> > > > > > > > > Just double checking - you do have space on the logs:
> > > > > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > > > > into.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Boaz
> > > > > > > > >
> > > > > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > > > > >
> > > > > > > > > > I installed from tar.
> > > > > > > > > >
> > > > > > > > > > my configuration.yml looks like this:
> > > > > > > > > >
> > > > > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > > > SCluster
> > > > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > > > "SCNode1Master"
> > > > > > > > > >
> > > > > > > > > > node.master: true
> > > > > > > > > > index.number\_of\_shards: 8
> > > > > > > > > >
> > > > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > > > path:
> > > > > > > > > > logs: /var/log/elasticsearch
> > > > > > > > > >
> > > > > > > > > > cloud:
> > > > > > > > > > aws:
> > > > > > > > > > access\_key: XXXXXX
> > > > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > > > discovery:
> > > > > > > > > > type: ec2
> > > > > > > > > > ec2:
> > > > > > > > > > groups: GROUP\_NAME
> > > > > > > > > >
> > > > > > > > > > cloud.node.auto\_attributes: true
> > > > > > > > > >
> > > > > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > > >
> > > > > > > > > > > Do you use the rpm/debia packages or is it an install from a tar
> > > > > > > > > > > ball/ zip? Can you share the elasticsearch.yml configuration? Also, this
> > > > > > > > > > > can potentially be overridden using command line parameters (but I assume
> > > > > > > > > > > you don't do that).
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > > > > >
> > > > > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > > > > >
> > > > > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi Daniel,
> > > > > > > > > > > > >
> > > > > > > > > > > > > It looks like you nodes do not discover each other - that
> > > > > > > > > > > > > number of nodes should be 3 for you.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > Boaz
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > so..
> > > > > > > > > > > > > > I decided to create a three nodes cluster (two data nodes and
> > > > > > > > > > > > > > one arbiter)
> > > > > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_ I
> > > > > > > > > > > > > > get the following response:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > {
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > > > > - status: "red",
> > > > > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > }
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > > > > - AND
> > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > > > > arbiter responding
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - status: "green"
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > - witch makes me belive that they are not really
> > > > > > > > > > > > > > connected to the same cluster..
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > > -
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > You received this message because you are subscribed to a topic
> > > > > > > > > > > > > > in the Google Groups "elasticsearch" group.
> > > > > > > > > > > > > > To unsubscribe from this topic, visit
> > > > > > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/)
> > > > > > > > > > > > > > unsubscribe.
> > > > > > > > > > > > > > To unsubscribe from this group and all its topics, send an
> > > > > > > > > > > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > > > For more options, visit [https://groups.google.com/grou](https://groups.google.com/grou)
> > > > > > > > > > > > > > ps/opt\_out.
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > > > > Groups "elasticsearch" group.
> > > > > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > > >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to the Google
> > > > > > > Groups "elasticsearch" group.
> > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > > > Groups "elasticsearch" group.
> > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > >
> > > --
> > > You received this message because you are subscribed to a topic in the
> > > Google Groups "elasticsearch" group.
> > > To unsubscribe from this topic, visit
> > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > > To unsubscribe from this group and all its topics, send an email to
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
[https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
To unsubscribe from this group and all its topics, send an email to
[elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Norberto\_Meijome](https://discuss.elastic.co/u/Norberto_Meijome)
#### Post date: [November 27, 2013, 2:08am UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/18 "2013-11-27T02:08:39Z")
On ec2, multicast won't work, so you need unicast.
If you use the ec2 plugin (you DO want to use it 😉 ), disable zen
discovery, it seems the ec2 plugin only kicks in after zen had given up (
maybe in the versions I run, but seems pretty consistent).
On 26/11/2013 9:34 PM, "Daniel Rasta" [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com) wrote:
> Sounds great! But is there something I can use to verify my config file?
>
> Also, every other setting entry seems to work just fine..
>
> My main consideration is the shards allocation. Even with the unicast
> discovery, they should be allocated.
>
> Sent from my iPhone.
> דניאל
> 0546350772
> [http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
>
> On 26 בנוב 2013, at 10:28, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
>
> Hi Daniel,
>
> ES will use 9300 for internal cross node communication (actually 9300-9400
> but that's only relevant if you run multiple nodes on a single server).
> 9200 is used for HTTP request to the API.
>
> Given the fact that the logging settings were not picked up I think the
> source of your problem is the settings file. I'd check it for correct
> indentation / structure.
>
> Cheers,
> Boaz
>
> On Tue, Nov 26, 2013 at 7:29 AM, Daniel Rasta [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com)wrote:
>
> > The logs were found on the es\_home directory, like you suggested.
> > And i did opened the ports (9200-9300) on the security group..
> >
> > Unicast is using the same port, no?
> >
> > Sent from my iPhone.
> > דניאל
> > 0546350772
> > [http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
> >
> > On 26 בנוב 2013, at 08:26, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
> >
> > Hi Daniel,
> >
> > The nodes need to be able to communicate with each other on port 9300 .
> > Did you set up a security group for the servers that allows access on port
> > 9300 between them?
> >
> > I think you have a configuration issue. You don't need to use unicast
> > discovery on ec2 but rather the discovery mechanism supplied by the aws
> > plugin. Let's first find the cause of that and then look at the unassigned
> > shards as they will probably be caused by the same thing.
> >
> > Where did you find the logs in the end?
> >
> > Cheers,
> > Boaz
> >
> > On Tuesday, November 26, 2013 12:30:30 AM UTC+1, DanielR wrote:
> >
> > > I've deleted all of my data and now i have "only" 24 unassigned\_shards..
> > >
> > > On Tuesday, November 26, 2013 1:27:46 AM UTC+2, Mark Walkom wrote:
> > >
> > > > It's red because you will have unassigned primaries.
> > > >
> > > > Given your previous issues you might want to reload the data into new
> > > > indexes and start afresh?
> > > >
> > > > Regards,
> > > > Mark Walkom
> > > >
> > > > Infrastructure Engineer
> > > > Campaign Monitor
> > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > >
> > > > On 26 November 2013 10:22, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > >
> > > > > ok.
> > > > > so i used unicast and let's say i can live with this..
> > > > >
> > > > > my cluster is acting all weird again...
> > > > >
> > > > > still a red status and unassigned\_shards is 48
> > > > >
> > > > > -
> > > > >
> > > > > {
> > > > >
> > > > > - cluster\_name: "SCluster",
> > > > > - status: "red",
> > > > > - timed\_out: false,
> > > > > - number\_of\_nodes: 3,
> > > > > - number\_of\_data\_nodes: 3,
> > > > > - active\_primary\_shards: 8,
> > > > > - active\_shards: 24,
> > > > > - relocating\_shards: 0,
> > > > > - initializing\_shards: 0,
> > > > > - unassigned\_shards: 48
> > > > >
> > > > > }
> > > > >
> > > > > in my log i can see an entry
> > > > > "dangling index, exists on local file system, but not in cluster
> > > > > metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
> > > > >
> > > > > and i can't create it on the next node because "
> > > > > IndexAlreadyExistsException"...
> > > > >
> > > > > so... does the red status means??
> > > > >
> > > > > On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> > > > >
> > > > > > Then that's the issue.
> > > > > >
> > > > > > I don't have experience with EC2 discovery, but you might want to try
> > > > > > unicast discovery to rule that out as the issue -
> > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > > > > > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> > > > > >
> > > > > > Regards,
> > > > > > Mark Walkom
> > > > > >
> > > > > > Infrastructure Engineer
> > > > > > Campaign Monitor
> > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > >
> > > > > > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > >
> > > > > > > Nope...
> > > > > > > Both the arbiter and the second node are flagged with "new\_master"
> > > > > > >
> > > > > > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > > > > > new\_master [Node3][\][inet[/:9300]], reason:
> > > > > > > zen-disco-join (elected\_as\_master)
> > > > > > >
> > > > > > > [2013-11-25 22:49:44,079][INFO][cluster.service]
> > > > > > > [Arbiter] new\_master [Arbiter][\][inet[/:9300]],
> > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > >
> > > > > > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > > > > > >
> > > > > > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > > > > > >
> > > > > > > > Browse through the logs on each of your nodes and see if you can
> > > > > > > > find references to the others joining the master.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Mark Walkom
> > > > > > > >
> > > > > > > > Infrastructure Engineer
> > > > > > > > Campaign Monitor
> > > > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > > > >
> > > > > > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > > > >
> > > > > > > > > ok 🙂
> > > > > > > > > found the logs
> > > > > > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > > > > > port 9300...
> > > > > > > > >
> > > > > > > > > so.. here it is:
> > > > > > > > >
> > > > > > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > > > > > publish\_address {inet[/:9300]}
> > > > > > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > > > > > [SCNode1Master] SCluster/\
> > > > > > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > > > > > publish\_address {inet[/:9200]}
> > > > > > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > > > > > ~
> > > > > > > > >
> > > > > > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > >
> > > > > > > > > > Hi Daniel,
> > > > > > > > > >
> > > > > > > > > > Just double checking - you do have space on the logs:
> > > > > > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > > > > > into.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > > Boaz
> > > > > > > > > >
> > > > > > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > > > > > >
> > > > > > > > > > > I installed from tar.
> > > > > > > > > > >
> > > > > > > > > > > my configuration.yml looks like this:
> > > > > > > > > > >
> > > > > > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > > > > SCluster
> > > > > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > > > > "SCNode1Master"
> > > > > > > > > > >
> > > > > > > > > > > node.master: true
> > > > > > > > > > > index.number\_of\_shards: 8
> > > > > > > > > > >
> > > > > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > > > > path:
> > > > > > > > > > > logs: /var/log/elasticsearch
> > > > > > > > > > >
> > > > > > > > > > > cloud:
> > > > > > > > > > > aws:
> > > > > > > > > > > access\_key: XXXXXX
> > > > > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > > > > discovery:
> > > > > > > > > > > type: ec2
> > > > > > > > > > > ec2:
> > > > > > > > > > > groups: GROUP\_NAME
> > > > > > > > > > >
> > > > > > > > > > > cloud.node.auto\_attributes: true
> > > > > > > > > > >
> > > > > > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Do you use the rpm/debia packages or is it an install from a
> > > > > > > > > > > > tar ball/ zip? Can you share the elasticsearch.yml configuration? Also,
> > > > > > > > > > > > this can potentially be overridden using command line parameters (but I
> > > > > > > > > > > > assume you don't do that).
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Daniel,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > It looks like you nodes do not discover each other - that
> > > > > > > > > > > > > > number of nodes should be 3 for you.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > > Boaz
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > so..
> > > > > > > > > > > > > > > I decided to create a three nodes cluster (two data nodes
> > > > > > > > > > > > > > > and one arbiter)
> > > > > > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_I get the following response:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > > > > > - status: "red",
> > > > > > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > > > > > - AND
> > > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > > > > > arbiter responding
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - status: "green"
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - witch makes me belive that they are not really
> > > > > > > > > > > > > > > connected to the same cluster..
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > You received this message because you are subscribed to a
> > > > > > > > > > > > > > > topic in the Google Groups "elasticsearch" group.
> > > > > > > > > > > > > > > To unsubscribe from this topic, visit
> > > > > > > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/)
> > > > > > > > > > > > > > > unsubscribe.
> > > > > > > > > > > > > > > To unsubscribe from this group and all its topics, send an
> > > > > > > > > > > > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/grou](https://groups.google.com/grou)
> > > > > > > > > > > > > > > ps/opt\_out.
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > > > > > Groups "elasticsearch" group.
> > > > > > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > > > >
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > Groups "elasticsearch" group.
> > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit
> > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CACj2-4%2BwsGLF5cnVPmSSZJLTxdozU\_8pwpmCMr5bV6r5YahRpg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CACj2-4%2BwsGLF5cnVPmSSZJLTxdozU_8pwpmCMr5bV6r5YahRpg%40mail.gmail.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Boaz\_Leskes](https://discuss.elastic.co/u/Boaz_Leskes)
#### Post date: [November 27, 2013, 8:02am UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/19 "2013-11-27T08:02:05Z")
Hi Daniel,
You can post your elasticsearch.yml as a gist and I'll have a look.
As to the unassigned shards, what's your index settings? How many shards
and how many replicas?
Cheers,
B
On Tue, Nov 26, 2013 at 11:33 AM, Daniel Rasta [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com)wrote:
> Sounds great! But is there something I can use to verify my config file?
>
> Also, every other setting entry seems to work just fine..
>
> My main consideration is the shards allocation. Even with the unicast
> discovery, they should be allocated.
>
> Sent from my iPhone.
> דניאל
> 0546350772
> [http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
>
> On 26 בנוב 2013, at 10:28, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
>
> Hi Daniel,
>
> ES will use 9300 for internal cross node communication (actually 9300-9400
> but that's only relevant if you run multiple nodes on a single server).
> 9200 is used for HTTP request to the API.
>
> Given the fact that the logging settings were not picked up I think the
> source of your problem is the settings file. I'd check it for correct
> indentation / structure.
>
> Cheers,
> Boaz
>
> On Tue, Nov 26, 2013 at 7:29 AM, Daniel Rasta [danielrastaziv@gmail.com](mailto:danielrastaziv@gmail.com)wrote:
>
> > The logs were found on the es\_home directory, like you suggested.
> > And i did opened the ports (9200-9300) on the security group..
> >
> > Unicast is using the same port, no?
> >
> > Sent from my iPhone.
> > דניאל
> > 0546350772
> > [http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
> >
> > On 26 בנוב 2013, at 08:26, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
> >
> > Hi Daniel,
> >
> > The nodes need to be able to communicate with each other on port 9300 .
> > Did you set up a security group for the servers that allows access on port
> > 9300 between them?
> >
> > I think you have a configuration issue. You don't need to use unicast
> > discovery on ec2 but rather the discovery mechanism supplied by the aws
> > plugin. Let's first find the cause of that and then look at the unassigned
> > shards as they will probably be caused by the same thing.
> >
> > Where did you find the logs in the end?
> >
> > Cheers,
> > Boaz
> >
> > On Tuesday, November 26, 2013 12:30:30 AM UTC+1, DanielR wrote:
> >
> > > I've deleted all of my data and now i have "only" 24 unassigned\_shards..
> > >
> > > On Tuesday, November 26, 2013 1:27:46 AM UTC+2, Mark Walkom wrote:
> > >
> > > > It's red because you will have unassigned primaries.
> > > >
> > > > Given your previous issues you might want to reload the data into new
> > > > indexes and start afresh?
> > > >
> > > > Regards,
> > > > Mark Walkom
> > > >
> > > > Infrastructure Engineer
> > > > Campaign Monitor
> > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > >
> > > > On 26 November 2013 10:22, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > >
> > > > > ok.
> > > > > so i used unicast and let's say i can live with this..
> > > > >
> > > > > my cluster is acting all weird again...
> > > > >
> > > > > still a red status and unassigned\_shards is 48
> > > > >
> > > > > -
> > > > >
> > > > > {
> > > > >
> > > > > - cluster\_name: "SCluster",
> > > > > - status: "red",
> > > > > - timed\_out: false,
> > > > > - number\_of\_nodes: 3,
> > > > > - number\_of\_data\_nodes: 3,
> > > > > - active\_primary\_shards: 8,
> > > > > - active\_shards: 24,
> > > > > - relocating\_shards: 0,
> > > > > - initializing\_shards: 0,
> > > > > - unassigned\_shards: 48
> > > > >
> > > > > }
> > > > >
> > > > > in my log i can see an entry
> > > > > "dangling index, exists on local file system, but not in cluster
> > > > > metadata, scheduling to delete in [2h], auto import to cluster state [YES]"
> > > > >
> > > > > and i can't create it on the next node because "
> > > > > IndexAlreadyExistsException"...
> > > > >
> > > > > so... does the red status means??
> > > > >
> > > > > On Tuesday, November 26, 2013 12:59:19 AM UTC+2, Mark Walkom wrote:
> > > > >
> > > > > > Then that's the issue.
> > > > > >
> > > > > > I don't have experience with EC2 discovery, but you might want to try
> > > > > > unicast discovery to rule that out as the issue -
> > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/)
> > > > > > reference/current/modules-discovery-zen.html[http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ](http://www.google.com/url?q=http%3A%2F%2Fwww.elasticsearch.org%2Fguide%2Fen%2Felasticsearch%2Freference%2Fcurrent%2Fmodules-discovery-zen.html&sa=D&sntz=1&usg=AFQjCNG9-58KP15vouPfc1b49es6XbmwBQ)
> > > > > >
> > > > > > Regards,
> > > > > > Mark Walkom
> > > > > >
> > > > > > Infrastructure Engineer
> > > > > > Campaign Monitor
> > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > >
> > > > > > On 26 November 2013 09:57, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > >
> > > > > > > Nope...
> > > > > > > Both the arbiter and the second node are flagged with "new\_master"
> > > > > > >
> > > > > > > [2013-11-25 22:49:39,212][INFO][cluster.service] [Node3]
> > > > > > > new\_master [Node3][\][inet[/:9300]], reason:
> > > > > > > zen-disco-join (elected\_as\_master)
> > > > > > >
> > > > > > > [2013-11-25 22:49:44,079][INFO][cluster.service]
> > > > > > > [Arbiter] new\_master [Arbiter][\][inet[/:9300]],
> > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > >
> > > > > > > On Tuesday, November 26, 2013 12:53:59 AM UTC+2, Mark Walkom wrote:
> > > > > > >
> > > > > > > > 9300 is the port ES uses for intercluster comms, so that's fine.
> > > > > > > >
> > > > > > > > Browse through the logs on each of your nodes and see if you can
> > > > > > > > find references to the others joining the master.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Mark Walkom
> > > > > > > >
> > > > > > > > Infrastructure Engineer
> > > > > > > > Campaign Monitor
> > > > > > > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com)
> > > > > > > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com)[http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q](http://www.google.com/url?q=http%3A%2F%2Fwww.campaignmonitor.com&sa=D&sntz=1&usg=AFQjCNFv30c-WBiP6sfBmxXaWBP5YBZg1Q)
> > > > > > > >
> > > > > > > > On 26 November 2013 09:51, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com) wrote:
> > > > > > > >
> > > > > > > > > ok 🙂
> > > > > > > > > found the logs
> > > > > > > > > I think maybe cluster.service is trying to find the other nodes on
> > > > > > > > > port 9300...
> > > > > > > > >
> > > > > > > > > so.. here it is:
> > > > > > > > >
> > > > > > > > > [2013-11-25 21:56:08,072][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: initializing ...
> > > > > > > > > [2013-11-25 21:56:08,196][INFO][plugins]
> > > > > > > > > [SCNode1Master] loaded [cloud-aws], sites
> > > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: initialized
> > > > > > > > > [2013-11-25 21:56:13,667][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: starting ...
> > > > > > > > > [2013-11-25 21:56:13,849][INFO][transport]
> > > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9300]},
> > > > > > > > > publish\_address {inet[/:9300]}
> > > > > > > > > [2013-11-25 21:56:16,962][INFO][cluster.service]
> > > > > > > > > [SCNode1Master] new\_master [SCNode1Master][\][inet[/ter=true},
> > > > > > > > > reason: zen-disco-join (elected\_as\_master)
> > > > > > > > > [2013-11-25 21:56:17,014][INFO][discovery]
> > > > > > > > > [SCNode1Master] SCluster/\
> > > > > > > > > [2013-11-25 21:56:17,053][INFO][http]
> > > > > > > > > [SCNode1Master] bound\_address {inet[/0:0:0:0:0:0:0:0:9200]},
> > > > > > > > > publish\_address {inet[/:9200]}
> > > > > > > > > [2013-11-25 21:56:17,053][INFO][node]
> > > > > > > > > [SCNode1Master] {0.90.1}[926]: started
> > > > > > > > > [2013-11-25 21:56:17,114][INFO][gateway]
> > > > > > > > > [SCNode1Master] recovered [2] indices into cluster\_state
> > > > > > > > > ~
> > > > > > > > >
> > > > > > > > > On Monday, November 25, 2013 3:07:35 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > >
> > > > > > > > > > Hi Daniel,
> > > > > > > > > >
> > > > > > > > > > Just double checking - you do have space on the logs:
> > > > > > > > > > /var/log/elasticsearch? Also check for a logs folder where you untared ES
> > > > > > > > > > into.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > > Boaz
> > > > > > > > > >
> > > > > > > > > > On Friday, November 22, 2013 4:47:23 PM UTC+1, DanielR wrote:
> > > > > > > > > >
> > > > > > > > > > > I installed from tar.
> > > > > > > > > > >
> > > > > > > > > > > my configuration.yml looks like this:
> > > > > > > > > > >
> > > > > > > > > > > cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > > > > SCluster
> > > > > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > > > > "SCNode1Master"
> > > > > > > > > > >
> > > > > > > > > > > node.master: true
> > > > > > > > > > > index.number\_of\_shards: 8
> > > > > > > > > > >
> > > > > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > > > > path:
> > > > > > > > > > > logs: /var/log/elasticsearch
> > > > > > > > > > >
> > > > > > > > > > > cloud:
> > > > > > > > > > > aws:
> > > > > > > > > > > access\_key: XXXXXX
> > > > > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > > > > discovery:
> > > > > > > > > > > type: ec2
> > > > > > > > > > > ec2:
> > > > > > > > > > > groups: GROUP\_NAME
> > > > > > > > > > >
> > > > > > > > > > > cloud.node.auto\_attributes: true
> > > > > > > > > > >
> > > > > > > > > > > On Friday, November 22, 2013 3:53:16 PM UTC+2, Boaz Leskes wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Do you use the rpm/debia packages or is it an install from a
> > > > > > > > > > > > tar ball/ zip? Can you share the elasticsearch.yml configuration? Also,
> > > > > > > > > > > > this can potentially be overridden using command line parameters (but I
> > > > > > > > > > > > assume you don't do that).
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Nov 22, 2013 at 1:59 PM, DanielR [danielr...@gmail.com](mailto:danielr...@gmail.com)wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > ok.. thats embarrassing.. i cant locate the log file in
> > > > > > > > > > > > > "/var/log/elasticsearch" even that it's configured in the elasticsearch.yml
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Friday, November 22, 2013 2:39:42 PM UTC+2, Boaz Leskes
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Daniel,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > It looks like you nodes do not discover each other - that
> > > > > > > > > > > > > > number of nodes should be 3 for you.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I'd look at the logs for clues of why it fails.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Cheers,
> > > > > > > > > > > > > > Boaz
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Friday, November 22, 2013 1:35:05 PM UTC+1, DanielR wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > so..
> > > > > > > > > > > > > > > I decided to create a three nodes cluster (two data nodes
> > > > > > > > > > > > > > > and one arbiter)
> > > > > > > > > > > > > > > All three nodes are using ec2 discovery
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > but when i'm calling /\_cluster/health on the _master node_I get the following response:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > {
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - cluster\_name: "SCluster",
> > > > > > > > > > > > > > > - status: "red",
> > > > > > > > > > > > > > > - timed\_out: false,
> > > > > > > > > > > > > > > - number\_of\_nodes: 1,
> > > > > > > > > > > > > > > - number\_of\_data\_nodes: 1,
> > > > > > > > > > > > > > > - active\_primary\_shards: 0,
> > > > > > > > > > > > > > > - active\_shards: 0,
> > > > > > > > > > > > > > > - relocating\_shards: 0,
> > > > > > > > > > > > > > > - initializing\_shards: 0,
> > > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > }
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I can detect two troubling parameters:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - active\_primary\_shards: 0
> > > > > > > > > > > > > > > - AND
> > > > > > > > > > > > > > > - unassigned\_shards: 48
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > another troubling issue is that the other data node and the
> > > > > > > > > > > > > > > arbiter responding
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - status: "green"
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > - witch makes me belive that they are not really
> > > > > > > > > > > > > > > connected to the same cluster..
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > > -
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > You received this message because you are subscribed to a
> > > > > > > > > > > > > > > topic in the Google Groups "elasticsearch" group.
> > > > > > > > > > > > > > > To unsubscribe from this topic, visit
> > > > > > > > > > > > > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/)
> > > > > > > > > > > > > > > unsubscribe.
> > > > > > > > > > > > > > > To unsubscribe from this group and all its topics, send an
> > > > > > > > > > > > > > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > > > > For more options, visit [https://groups.google.com/grou](https://groups.google.com/grou)
> > > > > > > > > > > > > > > ps/opt\_out.
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > > > > > Groups "elasticsearch" group.
> > > > > > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > > > >
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to the Google
> > > > > > > > Groups "elasticsearch" group.
> > > > > > > > To unsubscribe from this group and stop receiving emails from it,
> > > > > > > > send an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > > >
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > > > Groups "elasticsearch" group.
> > > > > > To unsubscribe from this group and stop receiving emails from it, send
> > > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > >
> > > > --
> > > > You received this message because you are subscribed to a topic in the
> > > > Google Groups "elasticsearch" group.
> > > > To unsubscribe from this topic, visit
> > > > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > > > To unsubscribe from this group and all its topics, send an email to
> > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> >
> > --
> > You received this message because you are subscribed to a topic in the
> > Google Groups "elasticsearch" group.
> > To unsubscribe from this topic, visit
> > [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> > To unsubscribe from this group and all its topics, send an email to
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "elasticsearch" group.
> To unsubscribe from this topic, visit
> [https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/6cYLWYw6XHg/unsubscribe).
> To unsubscribe from this group and all its topics, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKzwz0ri6rZp29Gcn60LORhvyWr9cfcwCDSnMuz8qqvOoAX3zQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKzwz0ri6rZp29Gcn60LORhvyWr9cfcwCDSnMuz8qqvOoAX3zQ%40mail.gmail.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@danielr](https://discuss.elastic.co/u/danielr)
#### Post date: [November 27, 2013, 10:12am UTC](https://discuss.elastic.co/t/unassigned-shards-problam/14544/20 "2013-11-27T10:12:44Z")
Well.. My index has 8 shards and 2 replicas.
And the config looks like that:
cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
> > > > > > > > > > SCluster
> > > > > > > > > > node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
> > > > > > > > > > "SCNode1Master"
> > > > > > > > > >
> > > > > > > > > > node.master: true
> > > > > > > > > > index.number\_of\_shards: 8
> > > > > > > > > >
> > > > > > > > > > index.number\_of\_replicas: 2
> > > > > > > > > > path:
> > > > > > > > > > logs: /var/log/elasticsearch
> > > > > > > > > >
> > > > > > > > > > cloud:
> > > > > > > > > > aws:
> > > > > > > > > > access\_key: XXXXXX
> > > > > > > > > > secret\_key: XXXXXXXXX
> > > > > > > > > > discovery:
> > > > > > > > > > type: ec2
> > > > > > > > > > ec2:
> > > > > > > > > > groups: GROUP\_NAME
> > > > > > > > > >
> > > > > > > > > > cloud.node.auto\_attributes: true
10X!
Sent from my iPhone.
דניאל
0546350772
[http://il.linkedin.com/in/danielziv](http://il.linkedin.com/in/danielziv)
On 27 בנוב 2013, at 10:02, Boaz Leskes [b.leskes@gmail.com](mailto:b.leskes@gmail.com) wrote:
cluster.name[http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-\_rGgAY2He8KM6ZUy2TdG8g](http://www.google.com/url?q=http%3A%2F%2Fcluster.name&sa=D&sntz=1&usg=AFQjCNGbdss-_rGgAY2He8KM6ZUy2TdG8g):
SCluster
node.name[http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff\_rbggu0\_UXL6VsgDVF5NFCg](http://www.google.com/url?q=http%3A%2F%2Fnode.name&sa=D&sntz=1&usg=AFQjCNE8Cff_rbggu0_UXL6VsgDVF5NFCg):
"SCNode1Master"
node.master: true
index.number\_of\_shards: 8
index.number\_of\_replicas: 2
path:
logs: /var/log/elasticsearch
cloud:
aws:
access\_key: XXXXXX
secret\_key: XXXXXXXXX
discovery:
type: ec2
ec2:
groups: GROUP\_NAME
cloud.node.auto\_attributes: true
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3793539901095203409%40unknownmsgid](https://groups.google.com/d/msgid/elasticsearch/3793539901095203409%40unknownmsgid).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
[Next page](https://discuss.elastic.co/t/unassigned-shards-problam/14544.md?page=2)