# Unable to connect to ES Cluster on AWS Elasticsearch Service through Hadoop **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [January 14, 2016, 10:46pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283 "2016-01-14T22:46:01Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![shamak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shamak/32/7142_2.png) [@shamak](https://discuss.elastic.co/u/shamak)\ **Post date:** [January 14, 2016, 10:46pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/1 "2016-01-14T22:46:01Z")
I'm using version 2.2.0 of the elasticsearch-hadoop library. My ES cluster is hosted on Amazon Elasticsearch Service ([https://aws.amazon.com/elasticsearch-service/](https://aws.amazon.com/elasticsearch-service/)). There is only one exposed URL to hit the cluster ([https://myurl.com](https://myurl.com)). I've modified the access policy on AWS to allow requests from specific IPs and I can confirm this works because I am able to send a HTTP request and get the response back from the IPs. After this set up, in an Apache pig script is unable to connect to the instance through the given URL. The error is: ```auto org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot resolve ip for hostname: myurl.com at org.elasticsearch.hadoop.util.SettingsUtils.resolveHostToIpIfNecessary(SettingsUtils.java:78) at org.elasticsearch.hadoop.util.SettingsUtils.qualifyNodes(SettingsUtils.java:40) at org.elasticsearch.hadoop.util.SettingsUtils.declaredNodes(SettingsUtils.java:118) at org.elasticsearch.hadoop.util.SettingsUtils.discoveredOrDeclaredNodes(SettingsUtils.java:124) at org.elasticsearch.hadoop.rest.NetworkClient.(NetworkClient.java:58) at org.elasticsearch.hadoop.rest.RestClient.(RestClient.java:84) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:174) at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:378) at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:173) at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:149) at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:192) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98) at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:639) at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:285) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212) ``` I am able to send HTTP requests to the URL but I am unable to ping the URL itself, which explains why it is unable to resolve the IP. At this point, I'm not sure how I'm supposed to connect to the cluster on AWS. Any help would be greatly appreciated. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 15, 2016, 4:37pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/2 "2016-01-15T16:37:41Z")
@shaunak Formatted your post (again) to make it readable. Please do so yourself in the future - the easier it is to read, the higher the chances for a proper response. Your post doesn't contain any information about the setup - the stacktrace indicates your assessment is correct but that's about it. What does your PIG configuration looks like (the setup for the ES cluster)? Try turning on logging on the REST package as explained [here](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/logging.html) to see what connections are being made. Outside the official testing, there are several users that are using the integration against an ES cluster on AWS. Is your PIG job running locally or not? If you can send HTTP requests to the URL, do you have a proxy setup by any chance? What error does ping give? Have you tried `traceroute`/`tracert`? ---
**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\ **Post date:** [January 15, 2016, 4:49pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/3 "2016-01-15T16:49:11Z")
@costin I think you meant to tag @shamak 🙂 ---
**Author:** ![shamak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shamak/32/7142_2.png) [@shamak](https://discuss.elastic.co/u/shamak)\ **Post date:** [January 15, 2016, 6:11pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/4 "2016-01-15T18:11:52Z")
Apologies! I'll try to be more verbose in this post. I am running pig locally (pig -x local). I'm not sure what you mean by PIG configuration (setup for ES cluster) means? In the script, I make a call using the library using the command below. I don't have any other configuration as such. I am just trying to connect to the ES cluster using the library within PIG with the call here: ```auto STORE B INTO 'radio/artists' USING org.elasticsearch.hadoop.pig.EsStorage('es.nodes=https://url.com'); ``` Turning on logging in debug mode does not give any useful information, as far as I can tell. It writes down the version of the library it's using which is v.2.2.0 and ```auto DEBUG pig.EsStorage: Using pre-defined writer serializer [org.elasticsearch.hadoop.pig.PigValueWriter] as default ``` The error ping gives is 'cannot resolve [https://url.com](https://url.com): Unknown host. Traceroute gives an error saying the hostname is too long. Yes, there is a proxy but I don't believe it blocks outgoing ping requests. I tried using the direct IP of the URL I hit on AWS, and then in that case the library says 'Connection Timed out, retrying request' and ping also gives an output saying no response from the IP. I can't really use the IP because AWS is probably using a load balancer to direct requests to different hosts. Thanks for helping out. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 15, 2016, 6:31pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/5 "2016-01-15T18:31:47Z")
It looks like you have a network misconfiguration. Likely your proxy configuration makes the request work in browser (since it is the one using it) while the rest of the tools do not since they don't know the host itself. Getting the IP should not be a problem - AWS does use an external IP for the given host and changing it on each request is not really a good idea (though of course, it can happen). Investigate what is your proxy configuration in the browser and [configure](https://www.elastic.co/guide/en/elasticsearch/hadoop/master/configuration.html#_proxy) ES-Hadoop accordingly. Do note that you need to set the cloud/wan configuration as well (it doesn't seem to be in there). ---
**Author:** ![shamak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shamak/32/7142_2.png) [@shamak](https://discuss.elastic.co/u/shamak)\ **Post date:** [January 15, 2016, 6:41pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/6 "2016-01-15T18:41:07Z")
Also if it helps, I am running this on a RHEL VM on a desktop and then SSH-ing into it and running my scripts on it. Both machines are on the same network however (Stupid setup, don't ask why). So, I am able to curl on the command line as well to the host and I get a response from ES. Would it still be a network misconfiguration? ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 15, 2016, 7:01pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/7 "2016-01-15T19:01:06Z")
Likely your VM doesn't properly route requests outside the host or its DNS is misconfigured. This is typically the case with Hadoop VMs (many times one can't even download new packages onto it). Try to poke around and fix its DNS or rather nameserver to reach to the outside world - it should be able to do so without SSH or anything like that. Likely your distro documentation has some pointers. Any other solution would be way too complicated and based on what you write, it might be easier for you. ---
**Author:** ![shamak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shamak/32/7142_2.png) [@shamak](https://discuss.elastic.co/u/shamak)\ **Post date:** [January 15, 2016, 7:06pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/8 "2016-01-15T19:06:41Z")
Ok, so there was a problem with my URL. I had to remove the trailing slash at the end for it to resolve the IP. The host command correctly identifies the IP and so does elasticsearch-hadoop. But now, the problem is the library is timing out on the request and it says this: ```auto 16/01/15 11:03:08 INFO httpclient.HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out 16/01/15 11:03:08 INFO httpclient.HttpMethodDirector: Retrying request ``` Is this because the the data I'm writing may be too big to send over one request? Or does the library break it down into chunks? EDIT: Not because of size of data. I get the error even with an extremely small dataset. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 15, 2016, 7:44pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/9 "2016-01-15T19:44:05Z")
Your VM is misconfigured and doesn't allow internet access to the outside world. Try connecting to google, etc - forget about Pig, ES-Hadoop, etc.. Just try to connect from the VM to any public site - once that's working, you can resume. ---
**Author:** ![shamak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shamak/32/7142_2.png) [@shamak](https://discuss.elastic.co/u/shamak)\ **Post date:** [January 15, 2016, 7:50pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/10 "2016-01-15T19:50:29Z")
Works fine with any other public site. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [January 15, 2016, 7:51pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/11 "2016-01-15T19:51:42Z")
Can you connect to the ES hostname/port you want to? `telnet` and such? ---
**Author:** ![shamak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shamak/32/7142_2.png) [@shamak](https://discuss.elastic.co/u/shamak)\ **Post date:** [January 19, 2016, 10:22pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/12 "2016-01-19T22:22:34Z")
Sorry for the late reply, and yes I was able to connect by specifying the port for HTTP (80) or HTTPS (443). I did not catch the fact that it defaults to 9200 for the port. I also had to set it to wan.only.mode. Thanks for all the help. ---
**Author:** ![Maheb](https://avatars.discourse-cdn.com/v4/letter/m/51bf81/32.png) [@Maheb](https://discuss.elastic.co/u/Maheb)\ **Post date:** [March 21, 2016, 9:37pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/13 "2016-03-21T21:37:14Z")
Hi @shamak, Were you able to resolve this? I'm getting the same error when I try to connect to my ES cluster on AWS. ---
**Author:** ![shamak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shamak/32/7142_2.png) [@shamak](https://discuss.elastic.co/u/shamak)\ **Post date:** [March 21, 2016, 10:28pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/14 "2016-03-21T22:28:14Z")
Hey. Yes, if you specify the port and the IP address of your AWS cluster master node, it will work and wan.only mode set to true. ---
**Author:** ![Maheb](https://avatars.discourse-cdn.com/v4/letter/m/51bf81/32.png) [@Maheb](https://discuss.elastic.co/u/Maheb)\ **Post date:** [March 22, 2016, 5:49pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/15 "2016-03-22T17:49:10Z")
Thanks @shamak! Set wan.only.mode to true and it worked! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:25pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-es-cluster-on-aws-elasticsearch-service-through-hadoop/39283/16 "2017-07-06T13:25:35Z")