# Adding EC2 settings to elasticsearch.yml stops listening to port 9200

**URL:** <https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979>\
**Category:** Elasticsearch\
**Created:** [December 20, 2013, 9:04pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979 "2013-12-20T21:04:56Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 20, 2013, 9:04pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/1 "2013-12-20T21:04:56Z")

</div>

I am running ES 0.90.4 on an Ubuntu 12.04 LTS EC2 instance.

Starting this instance, it starts listening to port 9200:  
ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$ sudo lsof -i TCP | grep 9200  
java 1982 elasticsearch 102u IPv6 12069 0t0 TCP \*:9200 (LISTEN)  
ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$

If I add the EC2 plugin settings to /etc/elasticsearch/elasticsearch.yml  
and restart elasticsearch, it stops listening to port 9200:  
ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$ sudo lsof -i TCP | grep 9200  
ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$

If I yank the EC2 settings from /etc/elasticsearch/elasticsearch.yml and  
restart elasticsearch, it starts listening to 9200 again.

Is this expected behaviour? Does it start listening to a different port  
instead?

The settings that are added to /etc/elasticsearch/elasticsearch.yml are  
the settings mentioned in [http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/](http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/)

The settings are:

cluster.name: myuniquename-cluster  
cloud:  
aws:  
access\_key:   
secret\_key:   
discovery:  
type: ec2  
gateway:  
type: s3  
s3:  
bucket: myuniquename

Thanks!

- Steinar

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/upzchaa3mdxj.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/upzchaa3mdxj.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![simpsora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simpsora/32/647_2.png) [@simpsora](https://discuss.elastic.co/u/simpsora)\
**Post date:** [December 21, 2013, 2:47am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/2 "2013-12-21T02:47:50Z")

</div>

How long did you wait after starting with AWS settings before checking the  
lsof output? I've found that startup time on AWS can be quite slow, due to  
the fact that ES is describing all your instances and looking for other ES  
instances to cluster with. If you have a lot of instances, this could take  
quite a while.

I'd suggest enabling the logging as described here:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

Then watch the logs as the node comes up. There may be some clues there as  
to what's going on.

Ross

On Friday, 20 December 2013 14:04:56 UTC-7, Steinar Bang wrote:

> I am running ES 0.90.4 on an Ubuntu 12.04 LTS EC2 instance.
> 
> Starting this instance, it starts listening to port 9200:  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$ sudo lsof -i TCP | grep  
> 9200  
> java 1982 elasticsearch 102u IPv6 12069 0t0 TCP \*:9200  
> (LISTEN)  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$
> 
> If I add the EC2 plugin settings to /etc/elasticsearch/elasticsearch.yml  
> and restart elasticsearch, it stops listening to port 9200:  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$ sudo lsof -i TCP | grep  
> 9200  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$
> 
> If I yank the EC2 settings from /etc/elasticsearch/elasticsearch.yml and  
> restart elasticsearch, it starts listening to 9200 again.
> 
> Is this expected behaviour? Does it start listening to a different port  
> instead?
> 
> The settings that are added to /etc/elasticsearch/elasticsearch.yml are  
> the settings mentioned in  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/)
> 
> The settings are:
> 
> cluster.name: myuniquename-cluster  
> cloud:  
> aws:  
> access\_key:   
> secret\_key:   
> discovery:  
> type: ec2  
> gateway:  
> type: s3  
> s3:  
> bucket: myuniquename
> 
> Thanks!
> 
> - Steinar

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/123a7260-c8cd-4203-bd95-f796a5b3ea69%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/123a7260-c8cd-4203-bd95-f796a5b3ea69%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\
**Post date:** [December 21, 2013, 7:25am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/3 "2013-12-21T07:25:39Z")

</div>

The EC2 plugin doesn't mess with networking settings, so the behaviour  
you're describing is weird.

After ES starts, can you connect to it via `curl localhost:9200`?

Also, don't use the now deprecated S3 gateway, apart maybe from strictly  
testing purposes -- it's unsupported and unpredictable.

As suggested by Ross, maybe it takes time to ping all the instances in your  
AWS account -- do you set the `discovery.ec2.groups` settings to filter the  
list?

Karel

On Friday, December 20, 2013 10:04:56 PM UTC+1, Steinar Bang wrote:

> I am running ES 0.90.4 on an Ubuntu 12.04 LTS EC2 instance.
> 
> Starting this instance, it starts listening to port 9200:  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$ sudo lsof -i TCP | grep  
> 9200  
> java 1982 elasticsearch 102u IPv6 12069 0t0 TCP \*:9200  
> (LISTEN)  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$
> 
> If I add the EC2 plugin settings to /etc/elasticsearch/elasticsearch.yml  
> and restart elasticsearch, it stops listening to port 9200:  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$ sudo lsof -i TCP | grep  
> 9200  
> ubuntu@ip-172-31-31-127:/var/lib/elasticsearch$
> 
> If I yank the EC2 settings from /etc/elasticsearch/elasticsearch.yml and  
> restart elasticsearch, it starts listening to 9200 again.
> 
> Is this expected behaviour? Does it start listening to a different port  
> instead?
> 
> The settings that are added to /etc/elasticsearch/elasticsearch.yml are  
> the settings mentioned in  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/)
> 
> The settings are:
> 
> cluster.name: myuniquename-cluster  
> cloud:  
> aws:  
> access\_key:   
> secret\_key:   
> discovery:  
> type: ec2  
> gateway:  
> type: s3  
> s3:  
> bucket: myuniquename
> 
> Thanks!
> 
> - Steinar

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f07a9191-8fea-4660-a38c-85825c5237db%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f07a9191-8fea-4660-a38c-85825c5237db%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 21, 2013, 9:34am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/4 "2013-12-21T09:34:19Z")

</div>

> > > > > Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com):

> The EC2 plugin doesn't mess with networking settings, so the behaviour  
> you're describing is weird.

I got the same behaviour when /var/lib/elasticsearch/elasticsearch was a  
dangling symlink to /mnt/var/lib/elasticsearch/elasticsearch (from an  
experiment done in a previous run of the EC2 instance with indexing to  
/dev/xvdb ), so I thought it might be storage related...?

> After ES starts, can you connect to it via `curl localhost:9200`?

Well, I did  
telnet localhost 9200  
instead.

But no success there.

> Also, don't use the now deprecated S3 gateway, apart maybe from strictly  
> testing purposes -- it's unsupported and unpredictable.

Ah, Ok. Is there something else I could use instead when I run out of  
space on the instance itself? Because I fill up the disk on the  
instance pretty quickly.

I did an experiment indexing to the non-persistent /dev/xvdb "disk", and  
that worked fine wrt. to storage size and feeding rate. But this isn't  
something that can be used with EC2 instances that are stopped and  
started on demand.

> As suggested by Ross, maybe it takes time to ping all the instances in your  
> AWS account -- do you set the `discovery.ec2.groups` settings to filter the  
> list?

Right! I will try giving it more time.

Thanks!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87iouimtt0.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/87iouimtt0.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 21, 2013, 9:38am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/5 "2013-12-21T09:38:24Z")

</div>

> > > > > Ross Simpson [simpsora@gmail.com](mailto:simpsora@gmail.com):

> How long did you wait after starting with AWS settings before checking the  
> lsof output?

A minute or two or three, or maybe five. In any case much longer than  
the normal ES startup time on the EC2 instance.

> I've found that startup time on AWS can be quite slow, due to the fact  
> that ES is describing all your instances and looking for other ES  
> instances to cluster with. If you have a lot of instances, this could  
> take quite a while.

Ok, thanks! I will try giving it a bit longer time and see what happens.

> I'd suggest enabling the logging as described here:  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/)  
> Then watch the logs as the node comes up. There may be some clues there as  
> to what's going on.

Thanks for the heads up! Looking at the verbosity logging was my next  
angle of approach, becuse what I found in the logs wasn't very  
informative. It looked like a successful startup (except nothing was  
listening on port 9200).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87eh56mtm7.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/87eh56mtm7.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\
**Post date:** [December 21, 2013, 1:30pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/6 "2013-12-21T13:30:23Z")

</div>

> I got the same behaviour when /var/lib/elasticsearch/elasticsearch was a  
> dangling symlink (...) so I thought it might be storage related...?

Hmm, still, Elasticsearch should complain in the logs when the data dir is  
not accessible. Might be related to S3 gateway...

> Is there something else I could use instead when I run out of  
> space on the instance itself? Because I fill up the disk on the  
> instance pretty quickly.

Very good option on EC2 are EBS volumes, notably the IOPS (optimized or  
high I/O) ones. You can easily create snapshots, unmount the volume and  
mount it at different instance, etc.

(Some people also use ephemeral disks (which are _huge_ for reasonably  
large instances), and simply use lots of replicas to guard against data  
loss...)

Karel

P. S.

The [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/) tutorial,  
though quite old, is well worth thorough reading for understanding all the  
mechanics on EC2. There's another  
tutorial, [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/deploying-elasticsearch-with-chef-solo/),  
which automates many of the steps with Chef.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c33370fe-7ec4-4a2a-941a-b77cd8a3203c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c33370fe-7ec4-4a2a-941a-b77cd8a3203c%40googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 22, 2013, 5:02pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/7 "2013-12-22T17:02:19Z")

</div>

> > > > > Steinar Bang [sb@dod.no](mailto:sb@dod.no):  
> > > > > Ross Simpson [simpsora@gmail.com](mailto:simpsora@gmail.com):

> Ok, thanks! I will try giving it a bit longer time and see what  
> happens.

I gave it 1h 40minutes last night, but all that came out in the log file  
was:  
[2013-12-21 19:39:42,205][INFO][node] [Stem Cell] version[0.90.7], pid[1317], build[36897d0/2013-11-13T12:06:54Z]  
[2013-12-21 19:39:42,205][INFO][node] [Stem Cell] initializing ...  
[2013-12-21 19:39:42,229][INFO][plugins] [Stem Cell] loaded [river-datagen, cloud-aws], sites [head]  
[2013-12-21 19:39:44,591][DEBUG][discovery.zen.ping.multicast] [Stem Cell] using group [224.2.2.4], with port [54328], ttl [3], and address [null]  
[2013-12-21 19:39:44,596][DEBUG][discovery.zen.ping.unicast] [Stem Cell] using initial hosts , with concurrent\_connects [10]

The last two lines were new after I changed the logging config. All of  
the lines were written immediately after startup, and nothing more was  
written.

At the end of the 1h40 minutes, nothing was listening on port 9200.

> > I'd suggest enabling the logging as described here:  
> > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/)  
> > Then watch the logs as the node comes up. There may be some clues  
> > there as to what's going on.

Here are the logging changes I did:

\*\*\* logging.yml 2013/12/21 19:21:25 1.1  
--- logging.yml 2013/12/22 16:50:32 1.2

* * *

\*\*\* 8,21 \*\*\*\*  
com.amazonaws: WARN

```
# gateway

```

! #gateway: DEBUG  
#index.gateway: DEBUG

```
# peer shard recovery
#indices.recovery: DEBUG

# discovery

```

! #discovery: TRACE

```
index.search.slowlog: TRACE, index_search_slow_log_file
index.indexing.slowlog: TRACE, index_indexing_slow_log_file

```

--- 8,22 ----  
com.amazonaws: WARN

```
# gateway

```

! gateway: DEBUG  
#index.gateway: DEBUG

- org.apache: WARN

! discovery: TRACE

```
index.search.slowlog: TRACE, index_search_slow_log_file
index.indexing.slowlog: TRACE, index_indexing_slow_log_file

```

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/upzc8uvcn7j8.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/upzc8uvcn7j8.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 22, 2013, 6:45pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/8 "2013-12-22T18:45:51Z")

</div>

> > > > > Steinar Bang [sb@dod.no](mailto:sb@dod.no):

> I gave it 1h 40minutes last night, but all that came out in the log file  
> was:  
> [2013-12-21 19:39:42,205][INFO][node] [Stem Cell] version[0.90.7], pid[1317], build[36897d0/2013-11-13T12:06:54Z]  
> [2013-12-21 19:39:42,205][INFO][node] [Stem Cell] initializing ...  
> [2013-12-21 19:39:42,229][INFO][plugins] [Stem Cell] loaded [river-datagen, cloud-aws], sites [head]  
> [2013-12-21 19:39:44,591][DEBUG][discovery.zen.ping.multicast] [Stem Cell] using group [224.2.2.4], with port [54328], ttl [3], and address [null]  
> [2013-12-21 19:39:44,596][DEBUG][discovery.zen.ping.unicast] [Stem Cell] using initial hosts , with concurrent\_connects [10]

> The last two lines were new after I changed the logging config. All of  
> the lines were written immediately after startup, and nothing more was  
> written.

I removed the S3 settings and after 1h30min nothing is still listening  
on port 9200.

One more line was written to the log (all lines immediately after  
startup):  
[2013-12-22 17:07:43,419][INFO][node] [Destiny] version[0.90.7], pid[1255], build[36897d0/2013-11-13T12:06:54Z]  
[2013-12-22 17:07:43,420][INFO][node] [Destiny] initializing ...  
[2013-12-22 17:07:43,444][INFO][plugins] [Destiny] loaded [river-datagen, cloud-aws], sites [head]  
[2013-12-22 17:07:45,813][DEBUG][discovery.zen.ping.multicast] [Destiny] using group [224.2.2.4], with port [54328], ttl [3], and address [null]  
[2013-12-22 17:07:45,818][DEBUG][discovery.zen.ping.unicast] [Destiny] using initial hosts , with concurrent\_connects [10]  
[2013-12-22 17:07:46,971][DEBUG][gateway.local] [Destiny] using initial\_shards [quorum], list\_timeout [30s]

I removed the EC2 discovery settings and the log filled up with lots of  
stuff, including attempted start of my river (failing because nothing  
was listening on the expected port). ES was listening on port 9200 when  
I tested.

I guess I will have to read the EC2 document closely and try to get a  
feeling for what it is supposed to work (amazon EC2 instances are a new  
experience for me).

Thanks!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/upzc4n60n2qo.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/upzc4n60n2qo.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 22, 2013, 6:50pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/9 "2013-12-22T18:50:18Z")

</div>

> > > > > Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com):

> Very good option on EC2 are EBS volumes, notably the IOPS (optimized  
> or high I/O) ones. You can easily create snapshots, unmount the volume  
> and mount it at different instance, etc.

Ok, I will study up on EBS volumes. Thanks!

> (Some people also use ephemeral disks (which are _huge_ for reasonably  
> large instances), and simply use lots of replicas to guard against  
> data loss...)

I only have access to a single EC2 instance for doing a small test, and  
need to stop and start it on demand, so unfortunately this is not an  
option.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/upzczjnslnyt.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/upzczjnslnyt.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 22, 2013, 6:53pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/10 "2013-12-22T18:53:22Z")

</div>

What is river-datagen?

Note that multicast should be disabled by cloud-aws plugin. So this line should not appear:

> using group [224.2.2.4], with port [54328], ttl [3], and address [null]

Wondering if you have something wrong in elasticsearch.yml file.

--  
David 😉  
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 22 déc. 2013 à 19:45, Steinar Bang [sb@dod.no](mailto:sb@dod.no) a écrit :

> > > > > Steinar Bang [sb@dod.no](mailto:sb@dod.no):

> I gave it 1h 40minutes last night, but all that came out in the log file  
> was:  
> [2013-12-21 19:39:42,205][INFO][node] [Stem Cell] version[0.90.7], pid[1317], build[36897d0/2013-11-13T12:06:54Z]  
> [2013-12-21 19:39:42,205][INFO][node] [Stem Cell] initializing ...  
> [2013-12-21 19:39:42,229][INFO][plugins] [Stem Cell] loaded [river-datagen, cloud-aws], sites [head]  
> [2013-12-21 19:39:44,591][DEBUG][discovery.zen.ping.multicast] [Stem Cell] using group [224.2.2.4], with port [54328], ttl [3], and address [null]  
> [2013-12-21 19:39:44,596][DEBUG][discovery.zen.ping.unicast] [Stem Cell] using initial hosts , with concurrent\_connects [10]

> The last two lines were new after I changed the logging config. All of  
> the lines were written immediately after startup, and nothing more was  
> written.

I removed the S3 settings and after 1h30min nothing is still listening  
on port 9200.

One more line was written to the log (all lines immediately after  
startup):  
[2013-12-22 17:07:43,419][INFO][node] [Destiny] version[0.90.7], pid[1255], build[36897d0/2013-11-13T12:06:54Z]  
[2013-12-22 17:07:43,420][INFO][node] [Destiny] initializing ...  
[2013-12-22 17:07:43,444][INFO][plugins] [Destiny] loaded [river-datagen, cloud-aws], sites [head]  
[2013-12-22 17:07:45,813][DEBUG][discovery.zen.ping.multicast] [Destiny] using group [224.2.2.4], with port [54328], ttl [3], and address [null]  
[2013-12-22 17:07:45,818][DEBUG][discovery.zen.ping.unicast] [Destiny] using initial hosts , with concurrent\_connects [10]  
[2013-12-22 17:07:46,971][DEBUG][gateway.local] [Destiny] using initial\_shards [quorum], list\_timeout [30s]

I removed the EC2 discovery settings and the log filled up with lots of  
stuff, including attempted start of my river (failing because nothing  
was listening on the expected port). ES was listening on port 9200 when  
I tested.

I guess I will have to read the EC2 document closely and try to get a  
feeling for what it is supposed to work (amazon EC2 instances are a new  
experience for me).

Thanks!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/upzc4n60n2qo.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/upzc4n60n2qo.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4A129CF1-6742-4884-BFB5-2DF29328755B%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/4A129CF1-6742-4884-BFB5-2DF29328755B%40pilato.fr).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 22, 2013, 9:43pm UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/11 "2013-12-22T21:43:09Z")

</div>

> > > > > David Pilato [david@pilato.fr](mailto:david@pilato.fr):

> Wondering if you have something wrong in elasticsearch.yml file.

Except for adding the EC2 stuff from the EC2 page, it is the file of the  
.deb package:  
[Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/tutorials/elasticsearch-on-ec2/)

Also, if I don't attempt to use either EC2 discovery or S3 gateway, ES  
works fine.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87lhzclfyq.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/87lhzclfyq.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 23, 2013, 1:09am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/12 "2013-12-23T01:09:13Z")

</div>

So there is probably something wrong with your elasticsearch.yml file.  
Could you gist it?

Preserve formating, just replace aws credentials with XXXX.

--  
David 😉  
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 22 déc. 2013 à 22:43, Steinar Bang [sb@dod.no](mailto:sb@dod.no) a écrit :

> > > > > David Pilato [david@pilato.fr](mailto:david@pilato.fr):

> Wondering if you have something wrong in elasticsearch.yml file.

Except for adding the EC2 stuff from the EC2 page, it is the file of the  
.deb package:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

Also, if I don't attempt to use either EC2 discovery or S3 gateway, ES  
works fine.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87lhzclfyq.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/87lhzclfyq.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/659CF695-EEC7-4B83-9AAA-BA0F7C507FD6%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/659CF695-EEC7-4B83-9AAA-BA0F7C507FD6%40pilato.fr).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 23, 2013, 10:00am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/13 "2013-12-23T10:00:20Z")

</div>

> > > > > David Pilato [david@pilato.fr](mailto:david@pilato.fr):

> So there is probably something wrong with your elasticsearch.yml file.  
> Could you gist it?

> <https://gist.github.com/steinarb/8094353>

Everything seems to be commented out, except for the stuff I've added in  
from the EC2 document...?

> Preserve formating, just replace aws credentials with XXXX.

Done. I also switched the name of the cluster and the S3 bucket.

Leaving in place either of the clustring settings or the S3 bucket  
settings causes the startup problems.

Could the problem be a mismatch between the EC2 plugin and the current  
ES...?

Oh, and I'm using OpenJDK instead of Oracle Java. Is that a potential  
complication?

Thanks!

- Steinar

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87a9frlwej.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/87a9frlwej.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [December 23, 2013, 10:50am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/14 "2013-12-23T10:50:53Z")

</div>

Could you try to add to elasticsearch.yml file:

cloud.enabled: true

and restart?

Also, your logs show that you are using elasticsearch 0.90.7, right?  
Did you install was plugin 1.16.0?

bin/plugin -install elasticsearch/elasticsearch-cloud-aws/1.16.0

--  
David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com)  
@dadoonet | @elasticsearchfr

Le 23 décembre 2013 at 11:00:35, Steinar Bang (sb@dod.no) a écrit:

> > > > > David Pilato [david@pilato.fr](mailto:david@pilato.fr):

> So there is probably something wrong with your elasticsearch.yml file.  
> Could you gist it?

> <https://gist.github.com/steinarb/8094353>

Everything seems to be commented out, except for the stuff I've added in  
from the EC2 document...?

> Preserve formating, just replace aws credentials with XXXX.

Done. I also switched the name of the cluster and the S3 bucket.

Leaving in place either of the clustring settings or the S3 bucket  
settings causes the startup problems.

Could the problem be a mismatch between the EC2 plugin and the current  
ES...?

Oh, and I'm using OpenJDK instead of Oracle Java. Is that a potential  
complication?

Thanks!

- Steinar

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87a9frlwej.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/87a9frlwej.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/etPan.52b8158d.5ff87e05.111%40MacBook-Air-de-David.local](https://groups.google.com/d/msgid/elasticsearch/etPan.52b8158d.5ff87e05.111%40MacBook-Air-de-David.local).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Steinar\_Bang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steinar_bang/32/122285_2.png) [@Steinar\_Bang](https://discuss.elastic.co/u/Steinar_Bang)\
**Post date:** [December 24, 2013, 8:44am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/15 "2013-12-24T08:44:07Z")

</div>

> > > > > David Pilato [david@pilato.fr](mailto:david@pilato.fr):

> Also, your logs show that you are using elasticsearch 0.90.7, right?

Correct.

> Did you install was plugin 1.16.0?

No, I had aws plugin 1.4.0.

> bin/plugin -install elasticsearch/elasticsearch-cloud-aws/1.16.0

I did that and now it runs and listens on port 9200 with the EC2  
clustring and S3 gateway settings in place.

Thanks!

(actually I first uninstalled 1.4.0. I don't know if that's necessary  
or not).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/87eh52iqp4.fsf%40dod.no](https://groups.google.com/d/msgid/elasticsearch/87eh52iqp4.fsf%40dod.no).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:59am UTC](https://discuss.elastic.co/t/adding-ec2-settings-to-elasticsearch-yml-stops-listening-to-port-9200/14979/16 "2017-07-06T01:59:32Z")

</div>