Elasticsearch not dynamically discovering ec2 instances


(Jay Zalowitz) #1

I am currently trying to set up an elasticsearch ami that looks across a
specific security groups instances and trys to add itself to the cluster,
as I need to be able to grow quickly as I run out of memory and speed up
requests as we launch.

ES is not reaching out to instances in the security group, any advice?

elasticsearch.yaml

    cluster.name: fetch
    cloud.aws.access_key: hidden
    cloud.aws.secret_key: hidden
    discovery.type: ec2
    discovery.ec2.ping_timeout: 1m
    discovery.zen.ping_timeout: 1m
    discovery.ec2.groups: ElasticSearch
    gateway.type: s3
    gateway.s3.bucket: bucketname
    gateway.recover_after_nodes: 2
    gateway.recover_after_time: 1m

Log:

        [2013-12-29 04:40:02,245][INFO ][node                     ] [Tutinax the Mountain-Mover] version[0.90.9], pid[1287], build[a968646/2013-12-23T10:35:28Z]
        [2013-12-29 04:40:02,246][INFO ][node                     ] [Tutinax the Mountain-Mover] initializing ...
        [2013-12-29 04:40:02,261][INFO ][plugins                  ] [Tutinax the Mountain-Mover] loaded [cloud-aws], sites []
        [2013-12-29 04:40:05,738][WARN ][gateway.s3               ] [Tutinax the Mountain-Mover] shared gateway has been deprecated, please use the (default) local gateway
        [2013-12-29 04:40:05,738][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] using bucket [bucketname], region [null], chunk_size [100mb], concurrent_streams [5]
        [2013-12-29 04:40:06,103][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] Latest metadata found at index [2]
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] initialized
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] starting ...
        [2013-12-29 04:40:06,278][INFO ][transport                ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/10.207.xx.xx:9301]}
        [2013-12-29 04:40:36,288][WARN ][discovery                ] [Tutinax the Mountain-Mover] waited for 30s and no initial state was set by the discovery
        [2013-12-29 04:40:36,288][INFO ][discovery                ] [Tutinax the Mountain-Mover] fetch/Eb5OByyGTsuwLrtRbJNGjw
        [2013-12-29 04:40:36,288][DEBUG][gateway                  ] [Tutinax the Mountain-Mover] can't wait on start for (possibly) reading state from gateway, will do it asynchronously
        [2013-12-29 04:40:36,294][INFO ][http                     ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9201]}, publish_address {inet[/10.207.xx.xx:9201]}
        [2013-12-29 04:40:36,295][INFO ][node                     ] [Tutinax the Mountain-Mover] started

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f83873a7-804d-4c99-964f-3b6353ee9849%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Frederick Cheung) #2

On Sunday, December 29, 2013 5:41:20 PM UTC, Jay Zalowitz wrote:

I am currently trying to set up an elasticsearch ami that looks across a
specific security groups instances and trys to add itself to the cluster,
as I need to be able to grow quickly as I run out of memory and speed up
requests as we launch.

ES is not reaching out to instances in the security group, any advice?

Have you set the EC2 region?

Fred

elasticsearch.yaml

    cluster.name: fetch
    cloud.aws.access_key: hidden
    cloud.aws.secret_key: hidden
    discovery.type: ec2
    discovery.ec2.ping_timeout: 1m
    discovery.zen.ping_timeout: 1m
    discovery.ec2.groups: ElasticSearch
    gateway.type: s3
    gateway.s3.bucket: bucketname
    gateway.recover_after_nodes: 2
    gateway.recover_after_time: 1m

Log:

        [2013-12-29 04:40:02,245][INFO ][node                     ] [Tutinax the Mountain-Mover] version[0.90.9], pid[1287], build[a968646/2013-12-23T10:35:28Z]
        [2013-12-29 04:40:02,246][INFO ][node                     ] [Tutinax the Mountain-Mover] initializing ...
        [2013-12-29 04:40:02,261][INFO ][plugins                  ] [Tutinax the Mountain-Mover] loaded [cloud-aws], sites []
        [2013-12-29 04:40:05,738][WARN ][gateway.s3               ] [Tutinax the Mountain-Mover] shared gateway has been deprecated, please use the (default) local gateway
        [2013-12-29 04:40:05,738][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] using bucket [bucketname], region [null], chunk_size [100mb], concurrent_streams [5]
        [2013-12-29 04:40:06,103][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] Latest metadata found at index [2]
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] initialized
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] starting ...
        [2013-12-29 04:40:06,278][INFO ][transport                ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/10.207.xx.xx:9301]}
        [2013-12-29 04:40:36,288][WARN ][discovery                ] [Tutinax the Mountain-Mover] waited for 30s and no initial state was set by the discovery
        [2013-12-29 04:40:36,288][INFO ][discovery                ] [Tutinax the Mountain-Mover] fetch/Eb5OByyGTsuwLrtRbJNGjw
        [2013-12-29 04:40:36,288][DEBUG][gateway                  ] [Tutinax the Mountain-Mover] can't wait on start for (possibly) reading state from gateway, will do it asynchronously
        [2013-12-29 04:40:36,294][INFO ][http                     ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9201]}, publish_address {inet[/10.207.xx.xx:9201]}
        [2013-12-29 04:40:36,295][INFO ][node                     ] [Tutinax the Mountain-Mover] started

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f22be1db-6d26-4402-b46a-9e72396cf153%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jay Zalowitz) #3

Is that a requirement for it to work? Otherwise id rather default to the
security group.

On Sunday, December 29, 2013 10:36:04 AM UTC-8, Frederick Cheung wrote:

On Sunday, December 29, 2013 5:41:20 PM UTC, Jay Zalowitz wrote:

I am currently trying to set up an elasticsearch ami that looks across a
specific security groups instances and trys to add itself to the cluster,
as I need to be able to grow quickly as I run out of memory and speed up
requests as we launch.

ES is not reaching out to instances in the security group, any advice?

Have you set the EC2 region?

Fred

elasticsearch.yaml

    cluster.name: fetch
    cloud.aws.access_key: hidden
    cloud.aws.secret_key: hidden
    discovery.type: ec2
    discovery.ec2.ping_timeout: 1m
    discovery.zen.ping_timeout: 1m
    discovery.ec2.groups: ElasticSearch
    gateway.type: s3
    gateway.s3.bucket: bucketname
    gateway.recover_after_nodes: 2
    gateway.recover_after_time: 1m

Log:

        [2013-12-29 04:40:02,245][INFO ][node                     ] [Tutinax the Mountain-Mover] version[0.90.9], pid[1287], build[a968646/2013-12-23T10:35:28Z]
        [2013-12-29 04:40:02,246][INFO ][node                     ] [Tutinax the Mountain-Mover] initializing ...
        [2013-12-29 04:40:02,261][INFO ][plugins                  ] [Tutinax the Mountain-Mover] loaded [cloud-aws], sites []
        [2013-12-29 04:40:05,738][WARN ][gateway.s3               ] [Tutinax the Mountain-Mover] shared gateway has been deprecated, please use the (default) local gateway
        [2013-12-29 04:40:05,738][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] using bucket [bucketname], region [null], chunk_size [100mb], concurrent_streams [5]
        [2013-12-29 04:40:06,103][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] Latest metadata found at index [2]
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] initialized
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] starting ...
        [2013-12-29 04:40:06,278][INFO ][transport                ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/10.207.xx.xx:9301]}
        [2013-12-29 04:40:36,288][WARN ][discovery                ] [Tutinax the Mountain-Mover] waited for 30s and no initial state was set by the discovery
        [2013-12-29 04:40:36,288][INFO ][discovery                ] [Tutinax the Mountain-Mover] fetch/Eb5OByyGTsuwLrtRbJNGjw
        [2013-12-29 04:40:36,288][DEBUG][gateway                  ] [Tutinax the Mountain-Mover] can't wait on start for (possibly) reading state from gateway, will do it asynchronously
        [2013-12-29 04:40:36,294][INFO ][http                     ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9201]}, publish_address {inet[/10.207.xx.xx:9201]}
        [2013-12-29 04:40:36,295][INFO ][node                     ] [Tutinax the Mountain-Mover] started

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3b95b9d8-30af-4cf8-a5da-cb290743583a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Frederick Cheung) #4

On Sunday, December 29, 2013 6:48:17 PM UTC, Jay Zalowitz wrote:

Is that a requirement for it to work? Otherwise id rather default to the
security group.

I believe so - ec2 regions are almost entirely separate from each other-
for example you can have identically named groups in multiple regions

Fred

On Sunday, December 29, 2013 10:36:04 AM UTC-8, Frederick Cheung wrote:

On Sunday, December 29, 2013 5:41:20 PM UTC, Jay Zalowitz wrote:

I am currently trying to set up an elasticsearch ami that looks across a
specific security groups instances and trys to add itself to the cluster,
as I need to be able to grow quickly as I run out of memory and speed up
requests as we launch.

ES is not reaching out to instances in the security group, any advice?

Have you set the EC2 region?

Fred

elasticsearch.yaml

    cluster.name: fetch
    cloud.aws.access_key: hidden
    cloud.aws.secret_key: hidden
    discovery.type: ec2
    discovery.ec2.ping_timeout: 1m
    discovery.zen.ping_timeout: 1m
    discovery.ec2.groups: ElasticSearch
    gateway.type: s3
    gateway.s3.bucket: bucketname
    gateway.recover_after_nodes: 2
    gateway.recover_after_time: 1m

Log:

        [2013-12-29 04:40:02,245][INFO ][node                     ] [Tutinax the Mountain-Mover] version[0.90.9], pid[1287], build[a968646/2013-12-23T10:35:28Z]
        [2013-12-29 04:40:02,246][INFO ][node                     ] [Tutinax the Mountain-Mover] initializing ...
        [2013-12-29 04:40:02,261][INFO ][plugins                  ] [Tutinax the Mountain-Mover] loaded [cloud-aws], sites []
        [2013-12-29 04:40:05,738][WARN ][gateway.s3               ] [Tutinax the Mountain-Mover] shared gateway has been deprecated, please use the (default) local gateway
        [2013-12-29 04:40:05,738][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] using bucket [bucketname], region [null], chunk_size [100mb], concurrent_streams [5]
        [2013-12-29 04:40:06,103][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] Latest metadata found at index [2]
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] initialized
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] starting ...
        [2013-12-29 04:40:06,278][INFO ][transport                ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/10.207.xx.xx:9301]}
        [2013-12-29 04:40:36,288][WARN ][discovery                ] [Tutinax the Mountain-Mover] waited for 30s and no initial state was set by the discovery
        [2013-12-29 04:40:36,288][INFO ][discovery                ] [Tutinax the Mountain-Mover] fetch/Eb5OByyGTsuwLrtRbJNGjw
        [2013-12-29 04:40:36,288][DEBUG][gateway                  ] [Tutinax the Mountain-Mover] can't wait on start for (possibly) reading state from gateway, will do it asynchronously
        [2013-12-29 04:40:36,294][INFO ][http                     ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9201]}, publish_address {inet[/10.207.xx.xx:9201]}
        [2013-12-29 04:40:36,295][INFO ][node                     ] [Tutinax the Mountain-Mover] started

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3eb48964-e61f-4744-b9bc-4e6756529122%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jay Zalowitz) #5

That didnt help..

I went all the way down to just:
cluster.name: fetch
cloud.aws.access_key: hidden
cloud.aws.secret_key: hidden
discovery.type: ec2

and it still didn't work.
On Sunday, December 29, 2013 11:29:14 AM UTC-8, Frederick Cheung wrote:

On Sunday, December 29, 2013 6:48:17 PM UTC, Jay Zalowitz wrote:

Is that a requirement for it to work? Otherwise id rather default to the
security group.

I believe so - ec2 regions are almost entirely separate from each other-
for example you can have identically named groups in multiple regions

Fred

On Sunday, December 29, 2013 10:36:04 AM UTC-8, Frederick Cheung wrote:

On Sunday, December 29, 2013 5:41:20 PM UTC, Jay Zalowitz wrote:

I am currently trying to set up an elasticsearch ami that looks across
a specific security groups instances and trys to add itself to the cluster,
as I need to be able to grow quickly as I run out of memory and speed up
requests as we launch.

ES is not reaching out to instances in the security group, any advice?

Have you set the EC2 region?

Fred

elasticsearch.yaml

    cluster.name: fetch
    cloud.aws.access_key: hidden
    cloud.aws.secret_key: hidden
    discovery.type: ec2
    discovery.ec2.ping_timeout: 1m
    discovery.zen.ping_timeout: 1m
    discovery.ec2.groups: ElasticSearch
    gateway.type: s3
    gateway.s3.bucket: bucketname
    gateway.recover_after_nodes: 2
    gateway.recover_after_time: 1m

Log:

        [2013-12-29 04:40:02,245][INFO ][node                     ] [Tutinax the Mountain-Mover] version[0.90.9], pid[1287], build[a968646/2013-12-23T10:35:28Z]
        [2013-12-29 04:40:02,246][INFO ][node                     ] [Tutinax the Mountain-Mover] initializing ...
        [2013-12-29 04:40:02,261][INFO ][plugins                  ] [Tutinax the Mountain-Mover] loaded [cloud-aws], sites []
        [2013-12-29 04:40:05,738][WARN ][gateway.s3               ] [Tutinax the Mountain-Mover] shared gateway has been deprecated, please use the (default) local gateway
        [2013-12-29 04:40:05,738][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] using bucket [bucketname], region [null], chunk_size [100mb], concurrent_streams [5]
        [2013-12-29 04:40:06,103][DEBUG][gateway.s3               ] [Tutinax the Mountain-Mover] Latest metadata found at index [2]
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] initialized
        [2013-12-29 04:40:06,120][INFO ][node                     ] [Tutinax the Mountain-Mover] starting ...
        [2013-12-29 04:40:06,278][INFO ][transport                ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9301]}, publish_address {inet[/10.207.xx.xx:9301]}
        [2013-12-29 04:40:36,288][WARN ][discovery                ] [Tutinax the Mountain-Mover] waited for 30s and no initial state was set by the discovery
        [2013-12-29 04:40:36,288][INFO ][discovery                ] [Tutinax the Mountain-Mover] fetch/Eb5OByyGTsuwLrtRbJNGjw
        [2013-12-29 04:40:36,288][DEBUG][gateway                  ] [Tutinax the Mountain-Mover] can't wait on start for (possibly) reading state from gateway, will do it asynchronously
        [2013-12-29 04:40:36,294][INFO ][http                     ] [Tutinax the Mountain-Mover] bound_address {inet[/0:0:0:0:0:0:0:0:9201]}, publish_address {inet[/10.207.xx.xx:9201]}
        [2013-12-29 04:40:36,295][INFO ][node                     ] [Tutinax the Mountain-Mover] started

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8d106950-67be-41a7-bc08-9b65ae733031%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #6