# Found - Hadoop integration issue **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [February 20, 2016, 8:56am UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294 "2016-02-20T08:56:47Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![d3nyable](https://avatars.discourse-cdn.com/v4/letter/d/839c29/32.png) [@d3nyable](https://discuss.elastic.co/u/d3nyable)\ **Post date:** [February 20, 2016, 8:56am UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/1 "2016-02-20T08:56:47Z")
Hi there, I've just signed up a trial account with Found and attempt to inject data through HiveQL using the elasticsearch-hadoop library like so: ``` ADD JAR some/url/elasticsearch-hadoop-2.2.0.jar CREATE EXTERNAL TABLE es_event ( requestId STRING, ipaddress STRING) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES('es.resource' = 'test', 'es.index.auto.create' = 'true', 'es.nodes' = 'found-host:9200'; ``` Now every time I attempt to connect to Found, I always get **EsHadoopInvalidRequest: null** error. Looking a bit deeper into the code, the error seems to be caused by 404 Not Found response because I assume that the library attempts to hit the ES API using the ip address instead of the provided host url. Manually hitting the API using the ip address through the browser returns this: {"ok":false,"message":"Unknown cluster."} Can anyone help me with this issue please? ---
**Author:** ![igor\_k](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_k/32/1157_2.png) [@igor\_k](https://discuss.elastic.co/u/igor_k)\ **Post date:** [February 20, 2016, 5:08pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/2 "2016-02-20T17:08:40Z")
Hi @d3nyable, Can you try setting the `es.nodes` property to your cluster url? You need to replace the `found-host`. > [@On another topic](https://discuss.elastic.co/t/42293/1): > > 'es.nodes' = 'found-host:9200' Thanks, Igor ---
**Author:** ![d3nyable](https://avatars.discourse-cdn.com/v4/letter/d/839c29/32.png) [@d3nyable](https://discuss.elastic.co/u/d3nyable)\ **Post date:** [February 21, 2016, 12:10am UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/3 "2016-02-21T00:10:55Z")
I did set the es.nodes value correctly according to my cluster URL like so: `'es.nodes' = 'xxxxxxxx.us-east-1.aws.found.io:9200'` I didnt write it like that initially because I thought it was obvious enough and didn't see the point of exposing the real URL. Any other suggestions? ---
**Author:** ![igor\_k](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_k/32/1157_2.png) [@igor\_k](https://discuss.elastic.co/u/igor_k)\ **Post date:** [February 21, 2016, 5:23am UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/4 "2016-02-21T05:23:58Z")
Hi, The URL indeed looks ok. Do you use shield in your cluster? If so then you'll also need to provide elasticsearch user and password, as described in [Security | Elasticsearch for Apache Hadoop [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/security.html) > #### Username/Password > > Set these through `es.net.http.auth.user` and `es.net.http.auth.pass` properties. If this will not help, can please you provide first 6 characters of your cluster ID? We will check if there is anything wrong with the cluster. Thanks, Igor ---
**Author:** ![d3nyable](https://avatars.discourse-cdn.com/v4/letter/d/839c29/32.png) [@d3nyable](https://discuss.elastic.co/u/d3nyable)\ **Post date:** [February 21, 2016, 6:47am UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/5 "2016-02-21T06:47:39Z")
Hi @igor_k Hitting the cluster host URL (e.g. [http://xxxx.us-east-1.aws.found.io:9200/](http://xxxx.us-east-1.aws.found.io:9200/)) brings back result like so: `{ "status" : 200, "name" : "instance-0000000000", "cluster_name" : "xxxxx", "version" : { "number" : "1.7.5", "build_hash" : "xxxx", "build_timestamp" : "2016-02-02T09:55:30Z", "build_snapshot" : false, "lucene_version" : "4.10.4" }, "tagline" : "You Know, for Search" }` but when I hit the remote ip address instead (e.g. x.x.x.x:9200), it returns with: `{"ok":false,"message":"Unknown cluster."}` I think that is the cause of the issue, the elasticsearch-hadoop library resolves the URL in es.nodes parameter to its remote ip address and use it to make the HTTP request. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [February 21, 2016, 3:37pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/6 "2016-02-21T15:37:28Z")
For cloud/wan environments, es-hadoop 2.2 introduced the wan option. It works with ES 1.x and 2.x. Please see the es-hadoop docs for more info. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [February 21, 2016, 5:23pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/7 "2016-02-21T17:23:11Z")
Just for the record, here is the [link](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/cloud.html) ---
**Author:** ![d3nyable](https://avatars.discourse-cdn.com/v4/letter/d/839c29/32.png) [@d3nyable](https://discuss.elastic.co/u/d3nyable)\ **Post date:** [February 21, 2016, 6:09pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/8 "2016-02-21T18:09:30Z")
Hi @costin Yeah i know about that setting and had also tried turning it on with no luck. The library is still making http requests using the remote ip address instead of the host url. It is weird that Found does not like it and returns with 404 instead. FYI I also tried AWS new ES Service and it works fine. Hitting either the remote ip or the URL returns back 200 with the expected JSON body. ---
**Author:** ![d3nyable](https://avatars.discourse-cdn.com/v4/letter/d/839c29/32.png) [@d3nyable](https://discuss.elastic.co/u/d3nyable)\ **Post date:** [February 21, 2016, 9:25pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/9 "2016-02-21T21:25:19Z")
I actually found the cause of this issue, which is `SettingsUtils.resolveHostToIpIfNecessary()` as it attempts to resolve host URL to ip address. Making http request using the remote ip address upsets Found and returns 404 as the request is missing the host header. Is there a reason why such logic is done? ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [February 21, 2016, 11:09pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/10 "2016-02-21T23:09:21Z")
See [https://github.com/elastic/elasticsearch-hadoop/issues/640](https://github.com/elastic/elasticsearch-hadoop/issues/640) Basically hostnames or IPs represent the same thing - hosts. So when dealing with lookups, one needs to resolve them to one single form that can be used for comparison. Hostnames are notoriusly problematic since there can be multiple hostnames pointing to the same ip, there can be aliases, etc... So using the IP instead is a great way of making things simpler. @d3nyable A solution to your case would be making resolution optional and maybe even disabling it by default for wan cases. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [February 21, 2016, 11:11pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/11 "2016-02-21T23:11:15Z")
FTR, linking the github [issue](https://github.com/elastic/elasticsearch-hadoop/issues/640) to this post. ---
**Author:** ![d3nyable](https://avatars.discourse-cdn.com/v4/letter/d/839c29/32.png) [@d3nyable](https://discuss.elastic.co/u/d3nyable)\ **Post date:** [February 21, 2016, 11:48pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/12 "2016-02-21T23:48:24Z")
Should I make a pull request to disable the logic when wan setting is set to true? ---
**Author:** ![d3nyable](https://avatars.discourse-cdn.com/v4/letter/d/839c29/32.png) [@d3nyable](https://discuss.elastic.co/u/d3nyable)\ **Post date:** [February 21, 2016, 11:48pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/13 "2016-02-21T23:48:42Z")
Thanks for your help BTW. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [February 22, 2016, 6:49pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/14 "2016-02-22T18:49:59Z")
PR are always welcome. I'll probably get around it by next week (currently travelling). Cheers, ---
**Author:** ![shivam\_rastogi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shivam_rastogi/32/47660_2.png) [@shivam\_rastogi](https://discuss.elastic.co/u/shivam_rastogi)\ **Post date:** [February 20, 2017, 3:58pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/15 "2017-02-20T15:58:05Z")
Hi igor\_k, I am new to this ELK stack and need your help regarding the Native Authentication. While performing a ES-HIVE integration, per your response , we need to define "es.net.http.auth.user and es.net.http.auth.pass properties." as part of the table DDL. Quentions is how to secure the credentials as anyone having read access to table metadata/execution access for "show create table" can access the credentials. Any help/guidance in this regard is appreciated. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:22pm UTC](https://discuss.elastic.co/t/found-hadoop-integration-issue/42294/16 "2017-07-06T13:22:31Z")