# Elasticsearch.yml chanages 2.x to 5.x on AWS EC2 - Amazon linux **URL:** \ **Category:** Elasticsearch\ **Created:** [June 2, 2017, 9:19am UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041 "2017-06-02T09:19:34Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![murlockvoices](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/murlockvoices/32/18746_2.png) [@murlockvoices](https://discuss.elastic.co/u/murlockvoices)\ **Post date:** [June 2, 2017, 9:19am UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/1 "2017-06-02T09:19:34Z")
Dear Elastic, I've decided to try out the Elasticsearch 5.x version, since I use the 2.x versions of this awesome product. I started to install and setup on AWS EC2 Amazon linux instance, but I encountered a couple of changes in the `elasticsearch.yml` config file. Some of them seems undocumented or not easy to find how to implement the 2.x configs in order to work whit 5.x. I installed the `ec2-discovery` plugin for elasticsearch. Can someone help me which parts of my yml setup needs to be changed or may removed since deprecated. Here is my 2.x yml which works like a charm: ```auto cluster.name: ES_TestCluster node.name: es_test_01 cloud.aws.region: eu-central-1 network.host: [_ec2_, _local_] plugin.mandatory: cloud-aws Cloud.aws.access_key: ******************************** cloud.aws.secret_key: **************************************************** discovery.type: ec2 discovery.zen.ping.multicast.enabled: false discovery.ec2.tag.TagGroup: ES_instance cluster.routing.allocation.awareness.attributes: zone node.zone: zoneB cluster.routing.allocation.awareness.force.zone.values: zoneA,zoneB indices.fielddata.cache.size: 50% gateway.recover_after_time: 5m ``` Thank you. 🙂 ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 2, 2017, 9:24am UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/2 "2017-06-02T09:24:15Z")
What problems are you having with this? ---
**Author:** ![murlockvoices](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/murlockvoices/32/18746_2.png) [@murlockvoices](https://discuss.elastic.co/u/murlockvoices)\ **Post date:** [June 2, 2017, 9:39am UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/3 "2017-06-02T09:39:13Z")
Thanks for your reply 🙂 I have many problems, but first of all Elasticsearch not able to start whit this setting. Why do you asking for problems? I assume you know very well the setup process on AWS EC2 Amazon linux and for you at a glance seems everything perfectly normal, and it should run without any problem. In this case there are a few mysteries I have to solve including that why I can't start Elasticsearch with this settings after a perfectly clean install. (Without these setting Elasticsearch runs perfectly.) Unfortunately I'm not an expert, so any help would be awesome 🙂 Have a nice day 🙂 ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [June 2, 2017, 9:42am UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/4 "2017-06-02T09:42:12Z")
Did you read the documentation for 5.X for the plugin to see what is and isn't valid? > [@murlockvoices](#): > > Why do you asking for problems? > I assume you know very well the setup process on AWS EC2 Amazon linux That's why, and there's your mistake. I am not an expert with the EC2 discovery plugin and neither are many people here. We are happy to try to help you, but making assumptions doesn't let us do that. ---
**Author:** ![murlockvoices](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/murlockvoices/32/18746_2.png) [@murlockvoices](https://discuss.elastic.co/u/murlockvoices)\ **Post date:** [June 2, 2017, 9:55am UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/5 "2017-06-02T09:55:05Z")
As you can see [here](https://discuss.elastic.co/t/getting-aws-credentials-from-keystore-for-discovery-ec2-plugin/87958/5) David Pilato points out the documentation is not up to date. > [@Getting AWS credentials from keystore for discovery-ec2 plugin?](https://discuss.elastic.co/t/getting-aws-credentials-from-keystore-for-discovery-ec2-plugin/87958/5): > > I see. Sadly the documentation has not been updated yet. It needs to be: discovery.ec2.access\_key discovery.ec2.secret\_key cloud.aws.region should not be used anymore but discovery.ec2.endpoint needs to be set in elasticsearch.yml file. See [https://github.com/elastic/elasticsearch/blob/5.4/plugins/discovery-ec2/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2Service.java#L181-L188](https://github.com/elastic/elasticsearch/blob/5.4/plugins/discovery-ec2/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2Service.java#L181-L188) I hope this helps. cc @rjernst in case I'm saying something wrong slight_smile Sorry for make you upset, it wasn't my intention. I hope we can still be friends 🙂 This post going to help many people in the near future cos as you said there are not many ec2-discovery expert here, and if someone going to look for a solution with a similar problem whit any keywords which are in the title, they need to see a solution or any ideas regarding this problem. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 14, 2017, 8:20pm UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/6 "2017-06-14T20:20:44Z")
Some comments about your settings: - `Cloud.aws.access_key` should be `cloud.aws.access_key`. - Remove `discovery.zen.ping.multicast.enabled: false` which is not needed anymore - `node.zone` should become `node.attr.zone` About deprecated settings, you can keep them for now as they are or try to migrate them to secured settings. But other things are going to change in the future so I'd probably wait for the latest elasticsearch 5.x version when 6.0 final will be released so you will be able to prepare the migration. And I hope that everything will be documented at this point. As I wrote in: > [@Getting AWS credentials from keystore for discovery-ec2 plugin?](https://discuss.elastic.co/t/getting-aws-credentials-from-keystore-for-discovery-ec2-plugin/87958/5): > > I see. Sadly the documentation has not been updated yet. It needs to be: discovery.ec2.access\_key discovery.ec2.secret\_key cloud.aws.region should not be used anymore but discovery.ec2.endpoint needs to be set in elasticsearch.yml file. See [https://github.com/elastic/elasticsearch/blob/5.4/plugins/discovery-ec2/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2Service.java#L181-L188](https://github.com/elastic/elasticsearch/blob/5.4/plugins/discovery-ec2/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2Service.java#L181-L188) I hope this helps. cc @rjernst in case I'm saying something wrong slight_smile You will need to register the following secured settings: - `discovery.ec2.access_key` - `discovery.ec2.secret_key` And `cloud.aws.region` should not be used anymore but `discovery.ec2.endpoint` needs to be set in `elasticsearch.yml` file. I hope this helps. If you still have warning messages or errors, please share your logs so we can provide more help. ---
**Author:** ![murlockvoices](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/murlockvoices/32/18746_2.png) [@murlockvoices](https://discuss.elastic.co/u/murlockvoices)\ **Post date:** [June 18, 2017, 6:22pm UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/7 "2017-06-18T18:22:33Z")
Hi David, Thanks for your response and help 🙂 It was not easy but I finally managed to start Elasticsearch with Kibana on Amazon Linux. Here is my elasticsearch.yml: ``` cluster.name: ES5_TestCluster node.name: es5_test_01 network.host: [_ec2_, _local_] discovery.zen.hosts_provider: ec2 discovery.ec2.endpoint: eu-central-1 discovery.ec2.tag.TagGroup: ES_instance cluster.routing.allocation.awareness.attributes: zone node.attr.zone: zoneB cluster.routing.allocation.awareness.force.zone.values: zoneA,zoneB indices.fielddata.cache.size: 50% gateway.recover_after_time: 5m ``` Actually this works fine however in the log I got this: **[INFO][o.e.d.e.AwsEc2UnicastHostsProvider] [es5\_test\_01] Exception while retrieving instance list from AWS API: Unable to execute HTTP request: eu-central-1: Name or service not known** I was proactive again and made a little research. The exception is generated when it calls the **_describeInstances()_** aws client function. > I checked that whether I'm able to run a describe function in the CLI, so I tried running the following: **_aws ec2 describe-instances --region eu-central-1_** It returned all the information of my instances, so it worked perfectly. It means something not quite right with my setup. Than I saw this line in the above code: **_logger.debug("Full exception:", e);_** So I switched Elasticsearch to debug mode maybe I can get more information. ``` PUT /_cluster/settings { "transient" : { "logger._root" : "DEBUG" } } ``` Although the log is more detailed unfortunately I still get the same line without any more info 😕 Additionally I get two visible exceptions in the log: (I copied just the first lines) **[DEBUG][r.suppressed] path: /\_cluster/nodes/stats, params: {pretty=, index=\_cluster, id=stats, type=nodes, nodeId=stats}** **java.lang.IllegalArgumentException: request [/\_cluster/nodes/stats] contains unrecognized parameter: [nodeId]** **[DEBUG][r.suppressed] path: /nodes/stats, params: {pretty=, index=nodes, type=stats}** **java.lang.IllegalArgumentException: No endpoint or operation is available at [stats]** If you need the full log I can upload it. There is a part in the code where it should log some aws settings: > BUT it's completely missing from the log. 😕 Where can I switch on the proper debug mode in order to this function **_logger.isDebugEnabled()_** return true? Do you have any idea regarding these issues? Thank you for any help. 🙂 _/sorry for the long post/_ ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 30, 2017, 3:06pm UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/8 "2017-06-30T15:06:49Z")
The `discovery.ec2.endpoint` for `eu-central-1` region should be something like: `ec2.eu-central-1.amazonaws.com` IIRC or `https://ec2.eu-central-1.amazonaws.com` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 28, 2017, 3:07pm UTC](https://discuss.elastic.co/t/elasticsearch-yml-chanages-2-x-to-5-x-on-aws-ec2-amazon-linux/88041/9 "2017-07-28T15:07:04Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.