Adding a new node to a ES cluster

Elastic Stack Elasticsearch
1

Hi:
I have a single node Elasticsearch instance running (version 1.0.0).
This instance was configured
with multicast false and no unicast IPs specified. and I change the default
ports from 9200,9300 to 9600,9700 with 5 Shards and no replication.

I just added a new node to this instance like so :
on a new server, I used the same exact Elasticsearch version, with the
configuration file copied over from the above instance, modified the
unicast list of hosts to include the IP address of the above server
instance.

I started up the node instance with this configuration. I then checked on
the cluster and saw the new instance being reported as an additional node.
I then used curl to up the replication factor to 2

My questions are these :

  1. How long does it take for the data to be synchronized /distributed so
    that the data is available to be queried properly ?
  2. Is the process I follow above flawed ? (are there any issues with it and
    Can I recover by stopping the cluster and restarting them with proper
    configuration set)
  3. Prior to addition of the node, I was able to query for documents older
    than 35 days (now-35d) but after the addition, this data is not available.
    a query for match_all returns the right number of documents, except that
    the older documents do not seem to be query able . If the new node that was
    added goes away how is the data affected ?

Thanks for the responses and assists.

Cheers

Ramdev

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9c250aa6-86a7-412f-a4df-afca9850dfb5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

1 - Depends on how much data you have.
2 - Yes, two replicas will mean one will never be assigned. This is because
you have 2 nodes but 3 copies of the data. Set replica to just 1.
3 - That sounds very unusual. Have you tried to fetch one of these
documents via id?

On 11 December 2014 at 15:50, Ramdev Wudali agastya71@gmail.com wrote:

Hi:
I have a single node Elasticsearch instance running (version 1.0.0).
This instance was configured
with multicast false and no unicast IPs specified. and I change the
default ports from 9200,9300 to 9600,9700 with 5 Shards and no replication.

I just added a new node to this instance like so :
on a new server, I used the same exact Elasticsearch version, with the
configuration file copied over from the above instance, modified the
unicast list of hosts to include the IP address of the above server
instance.

I started up the node instance with this configuration. I then checked on
the cluster and saw the new instance being reported as an additional node.
I then used curl to up the replication factor to 2

My questions are these :

  1. How long does it take for the data to be synchronized /distributed so
    that the data is available to be queried properly ?
  2. Is the process I follow above flawed ? (are there any issues with it
    and Can I recover by stopping the cluster and restarting them with proper
    configuration set)
  3. Prior to addition of the node, I was able to query for documents older
    than 35 days (now-35d) but after the addition, this data is not available.
    a query for match_all returns the right number of documents, except that
    the older documents do not seem to be query able . If the new node that was
    added goes away how is the data affected ?

Thanks for the responses and assists.

Cheers

Ramdev

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9c250aa6-86a7-412f-a4df-afca9850dfb5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9c250aa6-86a7-412f-a4df-afca9850dfb5%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-uEF%2BqYQQOOS7M5swZEbh%3DVCEVVG6FhnYC0A%3DQUf5wcw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

Hi Mark:
Thanks, a few things were resolved.

  1. I was running into heap memory issues on the new node, and the Cluster
    state went form being Yellow to Green almost immediately.
  2. The problem with my query was not so much with lack of data or data not
    being replicated/copied over to the new node, But something see hokey with
    the date math.

The query I was using before :

{
"query": {
"bool": {
"must": [
{"range": {
"DOC_DATE": {
"from": "now-36d",
"to": "now-35d"
}
}}
]
}
}
}

this would result in a number of documents.

However after I added the new node: the same query would not return any
docs. But when I changed
it to

{
"query": {
"bool": {
"must": [
{"range": {
"DOC_DATE": {
"from": "2014-11-14T00:00:00Z",
"to": "2014-11-15T00:00:00Z"
}
}}
]
}
}
}

I got documents.

Now the question is whats the problem with my prior query (if any).

Thanks

Ramdev

On Thu, Dec 11, 2014 at 10:59 AM, Mark Walkom markwalkom@gmail.com wrote:

1 - Depends on how much data you have.
2 - Yes, two replicas will mean one will never be assigned. This is
because you have 2 nodes but 3 copies of the data. Set replica to just 1.
3 - That sounds very unusual. Have you tried to fetch one of these
documents via id?

On 11 December 2014 at 15:50, Ramdev Wudali agastya71@gmail.com wrote:

Hi:
I have a single node Elasticsearch instance running (version 1.0.0).
This instance was configured
with multicast false and no unicast IPs specified. and I change the
default ports from 9200,9300 to 9600,9700 with 5 Shards and no replication.

I just added a new node to this instance like so :
on a new server, I used the same exact Elasticsearch version, with the
configuration file copied over from the above instance, modified the
unicast list of hosts to include the IP address of the above server
instance.

I started up the node instance with this configuration. I then checked on
the cluster and saw the new instance being reported as an additional node.
I then used curl to up the replication factor to 2

My questions are these :

  1. How long does it take for the data to be synchronized /distributed so
    that the data is available to be queried properly ?
  2. Is the process I follow above flawed ? (are there any issues with it
    and Can I recover by stopping the cluster and restarting them with proper
    configuration set)
  3. Prior to addition of the node, I was able to query for documents older
    than 35 days (now-35d) but after the addition, this data is not available.
    a query for match_all returns the right number of documents, except that
    the older documents do not seem to be query able . If the new node that was
    added goes away how is the data affected ?

Thanks for the responses and assists.

Cheers

Ramdev

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9c250aa6-86a7-412f-a4df-afca9850dfb5%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/9c250aa6-86a7-412f-a4df-afca9850dfb5%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/WlFs-7E90VU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-uEF%2BqYQQOOS7M5swZEbh%3DVCEVVG6FhnYC0A%3DQUf5wcw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-uEF%2BqYQQOOS7M5swZEbh%3DVCEVVG6FhnYC0A%3DQUf5wcw%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGbqZ7jo8kc7WeWqvcg-rkz8AJRX_JrXe5ZcnoPPM%2Bp%3D%2BJy%3D_Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.