# ES not working on ec2 with auto discover **URL:** \ **Category:** Elasticsearch\ **Created:** [January 25, 2014, 2:37am UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411 "2014-01-25T02:37:55Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![David\_Montgomery](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@David\_Montgomery](https://discuss.elastic.co/u/David_Montgomery)\ **Post date:** [January 25, 2014, 2:37am UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/1 "2014-01-25T02:37:55Z")
Hi, Newbie for installing ES esp on ec2. I used the deb package to installing ubuntu 12.04 anyway..... here is the basics of how I install with my chef recipe version = '0.90.10' remote\_file "#{Chef::Config[:file\_cache\_path]}/elasticsearch-#{version}.deb" do source "[https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb](https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#%7Bversion%7D.deb)" action :create\_if\_missing end dpkg\_package "#{Chef::Config[:file\_cache\_path]}/elasticsearch-#{version}.deb" do action :install end bash "add\_aws" do cwd "/usr/share/elasticsearch" code \<\<-EOH git clone [https://github.com/elasticsearch/elasticsearch-cloud-aws.git](https://github.com/elasticsearch/elasticsearch-cloud-aws.git) bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1 EOH not\_if {File.exists?("elasticsearch-cloud-aws")} end Here is my log file additions discovery.type: ec2 cloud.aws.access\_key: \<%=@AWS\_ACCESS\_KEY\_ID %\> cloud.aws.secret\_key: \<%=@AWS\_SECRET\_ACCESS\_KEY %\> So...wow...ES will not work. this is what i get sudo service elasticsearch status - elasticsearch is not running I run for the command line this is what I get... /usr/share/elasticsearch/bin/elasticsearch -f log4j:WARN No appenders could be found for logger (node). log4j:WARN Please initialize the log4j system properly. log4j:WARN See [http://logging.apache.org/log4j/1.2/faq.html#noconfig](http://logging.apache.org/log4j/1.2/faq.html#noconfig) for more info. I look at the logs..nothing that points to an error. So..how do I get auto discovery to work onb ec2? Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 25, 2014, 6:34am UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/2 "2014-01-25T06:34:35Z")
Aws plugin 2.0.0.RC1 works only with Elasticsearch 1.0.0.RC1 and \>. Look at the README. -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 25 janv. 2014 à 03:37, David Montgomery [davidmontgomery@gmail.com](mailto:davidmontgomery@gmail.com) a écrit : Hi, Newbie for installing ES esp on ec2. I used the deb package to installing ubuntu 12.04 anyway..... here is the basics of how I install with my chef recipe version = '0.90.10' remote\_file "#{Chef::Config[:file\_cache\_path]}/elasticsearch-#{version}.deb" do source "[https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb](https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#%7Bversion%7D.deb)" action :create\_if\_missing end dpkg\_package "#{Chef::Config[:file\_cache\_path]}/elasticsearch-#{version}.deb" do action :install end bash "add\_aws" do cwd "/usr/share/elasticsearch" code \<\<-EOH git clone [https://github.com/elasticsearch/elasticsearch-cloud-aws.git](https://github.com/elasticsearch/elasticsearch-cloud-aws.git) bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1 EOH not\_if {File.exists?("elasticsearch-cloud-aws")} end Here is my log file additions discovery.type: ec2 cloud.aws.access\_key: \<%=@AWS\_ACCESS\_KEY\_ID %\> cloud.aws.secret\_key: \<%=@AWS\_SECRET\_ACCESS\_KEY %\> So...wow...ES will not work. this is what i get sudo service elasticsearch status - elasticsearch is not running I run for the command line this is what I get... /usr/share/elasticsearch/bin/elasticsearch -f log4j:WARN No appenders could be found for logger (node). log4j:WARN Please initialize the log4j system properly. log4j:WARN See [http://logging.apache.org/log4j/1.2/faq.html#noconfig](http://logging.apache.org/log4j/1.2/faq.html#noconfig) for more info. I look at the logs..nothing that points to an error. So..how do I get auto discovery to work onb ec2? Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/1C800F3A-66F2-4A24-A5D8-A093793B0CE0%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/1C800F3A-66F2-4A24-A5D8-A093793B0CE0%40pilato.fr). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![David\_Montgomery](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@David\_Montgomery](https://discuss.elastic.co/u/David_Montgomery)\ **Post date:** [January 25, 2014, 5:24pm UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/3 "2014-01-25T17:24:40Z")
Well. I use the 1.0.0.RC1 this is what I get. ES still not working. What is missing? I installed elasticsearch-cloud-aws. /var/log/elasticsearch# service elasticsearch start - Starting Elasticsearch Server ...done. root@aws-elasticsearch-east-development:/var/log/elasticsearch# service elasticsearch status - elasticsearch is not running this is what is in the log [2014-01-25 15:28:59,363][INFO][node] [Novs] version[1.0.0.RC1], pid[18046], build[c6155c5/2014-01-15T17:02:32Z] [2014-01-25 15:28:59,364][INFO][node] [Novs] initializing ... [2014-01-25 15:28:59,452][INFO][plugins] [Novs] loaded [cloud-aws], sites [2014-01-25 16:44:34,376][INFO][node] [Amatsu-Mikaboshi] version[1.0.0.RC1], pid[18564], build[c6155c5/2014-01-15T17:02:32Z] [2014-01-25 16:44:34,377][INFO][node] [Amatsu-Mikaboshi] initializing ... [2014-01-25 16:44:34,422][INFO][plugins] [Amatsu-Mikaboshi] loaded [cloud-aws], sites [2014-01-25 16:54:00,859][INFO][node] [Bob] version[1.0.0.RC1], pid[18826], build[c6155c5/2014-01-15T17:02:32Z] [2014-01-25 16:54:00,860][INFO][node] [Bob] initializing ... [2014-01-25 16:54:00,904][INFO][plugins] [Bob] loaded [cloud-aws], sites Here is the elasticserch.yml file cluster.name: adfadfafafelasticsearch path.data: /data/es/data discovery.type: ec2 cloud.aws.access\_key: \<%=@AWS\_ACCESS\_KEY\_ID %\> cloud.aws.secret\_key: \<%=@AWS\_SECRET\_ACCESS\_KEY %\> On Saturday, January 25, 2014 2:34:35 PM UTC+8, David Pilato wrote: > Aws plugin 2.0.0.RC1 works only with Elasticsearch 1.0.0.RC1 and \>. > > Look at the README. > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 25 janv. 2014 à 03:37, David Montgomery \<[davidmo...@gmail.com](mailto:davidmo...@gmail.com)\\> > a écrit : > > Hi, > Newbie for installing ES esp on ec2. > > I used the deb package to installing ubuntu 12.04 > > anyway..... > > here is the basics of how I install with my chef recipe > > version = '0.90.10' > remote\_file > "#{Chef::Config[:file\_cache\_path]}/elasticsearch-#{version}.deb" do > source " > [https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb](https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#%7Bversion%7D.deb) > " > action :create\_if\_missing > end > > dpkg\_package > "#{Chef::Config[:file\_cache\_path]}/elasticsearch-#{version}.deb" do > action :install > end > bash "add\_aws" do > cwd "/usr/share/elasticsearch" > code \<\<-EOH > git clone [GitHub - elastic/elasticsearch-cloud-aws: AWS Cloud Plugin for Elasticsearch](https://github.com/elasticsearch/elasticsearch-cloud-aws.git) > bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1 > EOH > not\_if {File.exists?("elasticsearch-cloud-aws")} > end > > Here is my log file additions > discovery.type: ec2 > cloud.aws.access\_key: \<%=@AWS\_ACCESS\_KEY\_ID %\> > cloud.aws.secret\_key: \<%=@AWS\_SECRET\_ACCESS\_KEY %\> > > So...wow...ES will not work. > > this is what i get > sudo service elasticsearch status > > - elasticsearch is not running > > I run for the command line this is what I get... > > /usr/share/elasticsearch/bin/elasticsearch -f > log4j:WARN No appenders could be found for logger (node). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See [Apache log4j 1.2 - Frequently Asked Technical Questions](http://logging.apache.org/log4j/1.2/faq.html#noconfig) for > more info. > > I look at the logs..nothing that points to an error. > > So..how do I get auto discovery to work onb ec2? > > Thanks > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8dfd0de7-53bd-49b6-87c9-3cc1abaeb654%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8dfd0de7-53bd-49b6-87c9-3cc1abaeb654%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [January 26, 2014, 9:38am UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/4 "2014-01-26T09:38:30Z")
> I use the 1.0.0.RC1 this is what I get. ES still not working. What is > missing? I installed elasticsearch-cloud-aws. I would double check, that you're using the matching versions of Elasticsearch and the Cloud AWS plugin. Check the files actually installed, you should see `elasticsearch-cloud-aws-2.0.0.RC1.jar` for 1.x there. Also, you can increase the logging for AWS related events to have more visibility into the process: [https://github.com/elasticsearch/elasticsearch/blob/master/config/logging.yml#L8](https://github.com/elasticsearch/elasticsearch/blob/master/config/logging.yml#L8) If the problems stay, can you provide info on which region you're using, check if the credentials are valid, what type of instance you're using, how do you set ES\_HEAP\_SIZE? Finally, in your Chef recipe, you don't have to clone the repository, `bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1` should be enough... Karel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/57bf42b8-cb1c-44ae-b439-335efc4e18db%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/57bf42b8-cb1c-44ae-b439-335efc4e18db%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![David\_Montgomery](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@David\_Montgomery](https://discuss.elastic.co/u/David_Montgomery)\ **Post date:** [January 26, 2014, 4:12pm UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/5 "2014-01-26T16:12:42Z")
Hi, 1. the credentials are valid. Its the same creds that I use attached a volume and create tags using the aws cookbook where the are stored in a data bag. 2. I am in the east region and using a m1.small 3. The file exists sudo find / -name elasticsearch-cloud-aws-2.0.0.RC1.jar /usr/share/elasticsearch/plugins/cloud-aws/elasticsearch-cloud-aws-2.0.0.RC1.jar 4. I did not have a problem with ES when not using aws for auto discovery. Is there another way to cluster machines on aws? Clearly there is an issues somewhere. If I have 2 or more machines..is there another way? Thanks On Sunday, January 26, 2014 5:38:30 PM UTC+8, Karel Minařík wrote: > I use the 1.0.0.RC1 this is what I get. ES still not working. What is > > > missing? I installed elasticsearch-cloud-aws. > > I would double check, that you're using the matching versions of > Elasticsearch and the Cloud AWS plugin. Check the files actually installed, > you should see `elasticsearch-cloud-aws-2.0.0.RC1.jar` for 1.x there. > > Also, you can increase the logging for AWS related events to have more > visibility into the process: > [https://github.com/elasticsearch/elasticsearch/blob/master/config/logging.yml#L8](https://github.com/elasticsearch/elasticsearch/blob/master/config/logging.yml#L8) > > If the problems stay, can you provide info on which region you're using, > check if the credentials are valid, what type of instance you're using, how > do you set ES\_HEAP\_SIZE? > > Finally, in your Chef recipe, you don't have to clone the repository, > `bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1` > should be enough... > > Karel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/9fbbd022-a77c-4783-b21d-22b73dcad592%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/9fbbd022-a77c-4783-b21d-22b73dcad592%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [January 26, 2014, 4:32pm UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/6 "2014-01-26T16:32:01Z")
> 1. I did not have a problem with ES when not using aws for auto discovery. Is there another way to cluster machines on aws? Clearly there is an issues somewhere. If I have 2 or more machines..is there another way? The AWS discovery mechanism is alive and kicking in most cases. Based on your description, you seem to not be able to _start_ Elasticsearch with the plugin installed. Is that the case? For the AWS discovery to work, you have to open port 9300-9400 among the Elasticsearch instances. Have you done so? Best way how to achieve that in EC2 is to have a dedicated security group, which opens these ports to "itself". Screenshot (of the old AWS console): [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/content/themes/elasticsearch-org/images/content/chef-solo/create-security-group.png) Karel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/C2044BDE-6090-44CC-BD32-029F9216C986%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/C2044BDE-6090-44CC-BD32-029F9216C986%40gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![David\_Montgomery](https://avatars.discourse-cdn.com/v4/letter/d/51bf81/32.png) [@David\_Montgomery](https://discuss.elastic.co/u/David_Montgomery)\ **Post date:** [January 26, 2014, 5:00pm UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/7 "2014-01-26T17:00:37Z")
Hi, Ok........problem solved....was not discovery., I was the attachment of the volume. This brings up another issue.....why did I not get an error? logger: # log action execution errors for easier debugging action: DEBUG I just removed this line form elasticsearch.yml: path.data: /data I attachment is formated as mkfs.ext3 and the volume exists. is not path.data: /data is method to specify where the data should be stored? Thanks On Monday, January 27, 2014 12:32:01 AM UTC+8, Karel Minařík wrote: > > 1. I did not have a problem with ES when not using aws for auto > > discovery. Is there another way to cluster machines on aws? Clearly there > > is an issues somewhere. If I have 2 or more machines..is there another > > way? > > The AWS discovery mechanism is alive and kicking in most cases. Based on > your description, you seem to not be able to _start_ Elasticsearch with the > plugin installed. Is that the case? > > For the AWS discovery to work, you have to open port 9300-9400 among the > Elasticsearch instances. Have you done so? > > Best way how to achieve that in EC2 is to have a dedicated security group, > which opens these ports to "itself". Screenshot (of the old AWS console): > [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/content/themes/elasticsearch-org/images/content/chef-solo/create-security-group.png) > > Karel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e636e0fd-406e-4286-8633-cf8db252be0c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e636e0fd-406e-4286-8633-cf8db252be0c%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [January 26, 2014, 5:11pm UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/8 "2014-01-26T17:11:15Z")
> Ok........problem solved....was not discovery., I was the attachment of the volume. This brings up another issue.....why did I not get an error? What do you mean by "attachment of the volume"? The volume was not attached, was attached somewhere else, there was a permissions problem? When Elasticsearch cannot write to the data location, it prints out an error in the log. ``` elasticsearch -D es.path.data=/not/exist ... - ElasticsearchIllegalStateException[Failed to obtain node lock, is the following location writable?: [/not/exist/elasticsearch]] ``` > is not path.data: /data is method to specify where the data should be stored? Yes, it is: [https://github.com/elasticsearch/elasticsearch/blob/master/config/elasticsearch.yml#L147](https://github.com/elasticsearch/elasticsearch/blob/master/config/elasticsearch.yml#L147) Karel -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/F72AC65E-36AE-433D-A340-AA3A5832ABA7%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/F72AC65E-36AE-433D-A340-AA3A5832ABA7%40gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:54am UTC](https://discuss.elastic.co/t/es-not-working-on-ec2-with-auto-discover/15411/9 "2017-07-06T01:54:44Z")