ES not working on ec2 with auto discover


(David Montgomery) #1

Hi,
Newbie for installing ES esp on ec2.

I used the deb package to installing ubuntu 12.04

anyway.....

here is the basics of how I install with my chef recipe

version = '0.90.10'
remote_file
"#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
source
"https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb"
action :create_if_missing
end

dpkg_package
"#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
action :install
end
bash "add_aws" do
cwd "/usr/share/elasticsearch"
code <<-EOH
git clone https://github.com/elasticsearch/elasticsearch-cloud-aws.git
bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
EOH
not_if {File.exists?("elasticsearch-cloud-aws")}
end

Here is my log file additions
discovery.type: ec2
cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID %>
cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY %>

So...wow...ES will not work.

this is what i get
sudo service elasticsearch status

  • elasticsearch is not running

I run for the command line this is what I get...

/usr/share/elasticsearch/bin/elasticsearch -f
log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.

I look at the logs..nothing that points to an error.

So..how do I get auto discovery to work onb ec2?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #2

Aws plugin 2.0.0.RC1 works only with Elasticsearch 1.0.0.RC1 and >.

Look at the README.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 25 janv. 2014 à 03:37, David Montgomery davidmontgomery@gmail.com a écrit :

Hi,
Newbie for installing ES esp on ec2.

I used the deb package to installing ubuntu 12.04

anyway.....

here is the basics of how I install with my chef recipe

version = '0.90.10'
remote_file "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
source "https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb"
action :create_if_missing
end

dpkg_package "#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
action :install
end
bash "add_aws" do
cwd "/usr/share/elasticsearch"
code <<-EOH
git clone https://github.com/elasticsearch/elasticsearch-cloud-aws.git
bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
EOH
not_if {File.exists?("elasticsearch-cloud-aws")}
end

Here is my log file additions
discovery.type: ec2
cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID %>
cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY %>

So...wow...ES will not work.

this is what i get
sudo service elasticsearch status

  • elasticsearch is not running

I run for the command line this is what I get...

/usr/share/elasticsearch/bin/elasticsearch -f
log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

I look at the logs..nothing that points to an error.

So..how do I get auto discovery to work onb ec2?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/1C800F3A-66F2-4A24-A5D8-A093793B0CE0%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(David Montgomery) #3

Well.

I use the 1.0.0.RC1 this is what I get. ES still not working. What is
missing? I installed elasticsearch-cloud-aws.

/var/log/elasticsearch# service elasticsearch start

  • Starting Elasticsearch Server
    ...done.
    root@aws-elasticsearch-east-development:/var/log/elasticsearch# service
    elasticsearch status
  • elasticsearch is not running

this is what is in the log

[2014-01-25 15:28:59,363][INFO ][node ] [Novs]
version[1.0.0.RC1], pid[18046], build[c6155c5/2014-01-15T17:02:32Z]
[2014-01-25 15:28:59,364][INFO ][node ] [Novs]
initializing ...
[2014-01-25 15:28:59,452][INFO ][plugins ] [Novs] loaded
[cloud-aws], sites []
[2014-01-25 16:44:34,376][INFO ][node ]
[Amatsu-Mikaboshi] version[1.0.0.RC1], pid[18564],
build[c6155c5/2014-01-15T17:02:32Z]
[2014-01-25 16:44:34,377][INFO ][node ]
[Amatsu-Mikaboshi] initializing ...
[2014-01-25 16:44:34,422][INFO ][plugins ]
[Amatsu-Mikaboshi] loaded [cloud-aws], sites []
[2014-01-25 16:54:00,859][INFO ][node ] [Bob]
version[1.0.0.RC1], pid[18826], build[c6155c5/2014-01-15T17:02:32Z]
[2014-01-25 16:54:00,860][INFO ][node ] [Bob]
initializing ...
[2014-01-25 16:54:00,904][INFO ][plugins ] [Bob] loaded
[cloud-aws], sites []

Here is the elasticserch.yml file

cluster.name: adfadfafafelasticsearch
path.data: /data/es/data
discovery.type: ec2
cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID %>
cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY %>

On Saturday, January 25, 2014 2:34:35 PM UTC+8, David Pilato wrote:

Aws plugin 2.0.0.RC1 works only with Elasticsearch 1.0.0.RC1 and >.

Look at the README.

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 25 janv. 2014 à 03:37, David Montgomery <davidmo...@gmail.com<javascript:>>
a écrit :

Hi,
Newbie for installing ES esp on ec2.

I used the deb package to installing ubuntu 12.04

anyway.....

here is the basics of how I install with my chef recipe

version = '0.90.10'
remote_file
"#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
source "
https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-#{version}.deb
"
action :create_if_missing
end

dpkg_package
"#{Chef::Config[:file_cache_path]}/elasticsearch-#{version}.deb" do
action :install
end
bash "add_aws" do
cwd "/usr/share/elasticsearch"
code <<-EOH
git clone https://github.com/elasticsearch/elasticsearch-cloud-aws.git
bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
EOH
not_if {File.exists?("elasticsearch-cloud-aws")}
end

Here is my log file additions
discovery.type: ec2
cloud.aws.access_key: <%=@AWS_ACCESS_KEY_ID %>
cloud.aws.secret_key: <%=@AWS_SECRET_ACCESS_KEY %>

So...wow...ES will not work.

this is what i get
sudo service elasticsearch status

  • elasticsearch is not running

I run for the command line this is what I get...

/usr/share/elasticsearch/bin/elasticsearch -f
log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.

I look at the logs..nothing that points to an error.

So..how do I get auto discovery to work onb ec2?

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a48c0e49-e956-402b-8ea3-cf7037dc9f0d%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8dfd0de7-53bd-49b6-87c9-3cc1abaeb654%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Karel Minarik) #4

I use the 1.0.0.RC1 this is what I get. ES still not working. What is
missing? I installed elasticsearch-cloud-aws.

I would double check, that you're using the matching versions of
Elasticsearch and the Cloud AWS plugin. Check the files actually installed,
you should see elasticsearch-cloud-aws-2.0.0.RC1.jar for 1.x there.

Also, you can increase the logging for AWS related events to have more
visibility into the
process: https://github.com/elasticsearch/elasticsearch/blob/master/config/logging.yml#L8

If the problems stay, can you provide info on which region you're using,
check if the credentials are valid, what type of instance you're using, how
do you set ES_HEAP_SIZE?

Finally, in your Chef recipe, you don't have to clone the repository,
bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
should be enough...

Karel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/57bf42b8-cb1c-44ae-b439-335efc4e18db%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Montgomery) #5

Hi,

  1. the credentials are valid. Its the same creds that I use attached a
    volume and create tags using the aws cookbook where the are stored in a
    data bag.

  2. I am in the east region and using a m1.small

  3. The file exists
    sudo find / -name elasticsearch-cloud-aws-2.0.0.RC1.jar
    /usr/share/elasticsearch/plugins/cloud-aws/elasticsearch-cloud-aws-2.0.0.RC1.jar

  4. I did not have a problem with ES when not using aws for auto discovery.
    Is there another way to cluster machines on aws? Clearly there is an
    issues somewhere. If I have 2 or more machines..is there another way?

Thanks

On Sunday, January 26, 2014 5:38:30 PM UTC+8, Karel Minařík wrote:

I use the 1.0.0.RC1 this is what I get. ES still not working. What is

missing? I installed elasticsearch-cloud-aws.

I would double check, that you're using the matching versions of
Elasticsearch and the Cloud AWS plugin. Check the files actually installed,
you should see elasticsearch-cloud-aws-2.0.0.RC1.jar for 1.x there.

Also, you can increase the logging for AWS related events to have more
visibility into the process:
https://github.com/elasticsearch/elasticsearch/blob/master/config/logging.yml#L8

If the problems stay, can you provide info on which region you're using,
check if the credentials are valid, what type of instance you're using, how
do you set ES_HEAP_SIZE?

Finally, in your Chef recipe, you don't have to clone the repository,
bin/plugin -install elasticsearch/elasticsearch-cloud-aws/2.0.0.RC1
should be enough...

Karel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9fbbd022-a77c-4783-b21d-22b73dcad592%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Karel Minarik) #6
  1. I did not have a problem with ES when not using aws for auto discovery. Is there another way to cluster machines on aws? Clearly there is an issues somewhere. If I have 2 or more machines..is there another way?

The AWS discovery mechanism is alive and kicking in most cases. Based on your description, you seem to not be able to start Elasticsearch with the plugin installed. Is that the case?

For the AWS discovery to work, you have to open port 9300-9400 among the Elasticsearch instances. Have you done so?

Best way how to achieve that in EC2 is to have a dedicated security group, which opens these ports to "itself". Screenshot (of the old AWS console): http://www.elasticsearch.org/content/themes/elasticsearch-org/images/content/chef-solo/create-security-group.png

Karel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/C2044BDE-6090-44CC-BD32-029F9216C986%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Montgomery) #7

Hi,

Ok........problem solved....was not discovery., I was the attachment of
the volume. This brings up another issue.....why did I not get an error?

logger:

log action execution errors for easier debugging

action: DEBUG

I just removed this line form elasticsearch.yml: path.data: /data

I attachment is formated as mkfs.ext3 and the volume exists.

is not path.data: /data is method to specify where the data should be
stored?

Thanks

On Monday, January 27, 2014 12:32:01 AM UTC+8, Karel Minařík wrote:

  1. I did not have a problem with ES when not using aws for auto
    discovery. Is there another way to cluster machines on aws? Clearly there
    is an issues somewhere. If I have 2 or more machines..is there another
    way?

The AWS discovery mechanism is alive and kicking in most cases. Based on
your description, you seem to not be able to start Elasticsearch with the
plugin installed. Is that the case?

For the AWS discovery to work, you have to open port 9300-9400 among the
Elasticsearch instances. Have you done so?

Best way how to achieve that in EC2 is to have a dedicated security group,
which opens these ports to "itself". Screenshot (of the old AWS console):
http://www.elasticsearch.org/content/themes/elasticsearch-org/images/content/chef-solo/create-security-group.png

Karel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e636e0fd-406e-4286-8633-cf8db252be0c%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Karel Minarik) #8

Ok........problem solved....was not discovery., I was the attachment of the volume. This brings up another issue.....why did I not get an error?

What do you mean by "attachment of the volume"? The volume was not attached, was attached somewhere else, there was a permissions problem? When Elasticsearch cannot write to the data location, it prints out an error in the log.

elasticsearch -D es.path.data=/not/exist
...
- ElasticsearchIllegalStateException[Failed to obtain node lock, is the following location writable?: [/not/exist/elasticsearch]]

is not path.data: /data is method to specify where the data should be stored?

Yes, it is: https://github.com/elasticsearch/elasticsearch/blob/master/config/elasticsearch.yml#L147

Karel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/F72AC65E-36AE-433D-A340-AA3A5832ABA7%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #9