# Elastic Cluster with Hadoop

**URL:** <https://discuss.elastic.co/t/elastic-cluster-with-hadoop/171860>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [March 12, 2019, 5:15am UTC](https://discuss.elastic.co/t/elastic-cluster-with-hadoop/171860 "2019-03-12T05:15:37Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![shikhasingh](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@shikhasingh](https://discuss.elastic.co/u/shikhasingh)\
**Post date:** [March 12, 2019, 5:15am UTC](https://discuss.elastic.co/t/elastic-cluster-with-hadoop/171860/1 "2019-03-12T05:15:37Z")

</div>

Hi Team,

I have created one node ES cluster and want to integrate with my hadoop cluster, we are using ES 6.6 and HDP 3 version. we want to integrate ES with hive and followed the given documents.  
But when try to create an external table after connecting to hive on same node where ES cluster is running it says "Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only' (state=08S01,code=1)"

Command which I ran is CREATE EXTERNAL TABLE elastic\_table(name string, ID int, profession string) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES('es.resource' = 'customer','es.nodes.wan.only' = 'true','es.index.auto.create'='TRUE');

I suspect this is due to our hadoop cluster is kerberised, could you please guide us to make ES cluster accessible to our hadoop cluster.  
Looking for response.

Thanks a lot,  
Shikha

---

<div class="post-metadata">

**Author:** ![shikhasingh](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@shikhasingh](https://discuss.elastic.co/u/shikhasingh)\
**Post date:** [March 12, 2019, 6:33am UTC](https://discuss.elastic.co/t/elastic-cluster-with-hadoop/171860/2 "2019-03-12T06:33:10Z")

</div>

I have found [https://docs.search-guard.com/latest/search-guard-installation#first-time-installation-full-cluster-restart](https://docs.search-guard.com/latest/search-guard-installation#first-time-installation-full-cluster-restart) and trying to install searchguard so that i can configure the kerberos for ES.

Have downloaded 24.1 version of SearchGuard for ES 6.6.1 from mentioned link --

> **[Security for Elasticsearch | Latest Search Guard Versions | Search Guard](https://docs.search-guard.com/latest/search-guard-versions)**
>
> The Search Guard Community Edition provides TLS encryption and index-level permissions on REST and transport for free.

When try to run " ./elasticsearch-plugin install -b /etc/elasticsearch-6.6.1/search-guard-6-6.6.1-24.1.zip" but getting below error :-

ERROR: Unknown plugin /etc/elasticsearch-6.6.1/search-guard-6-6.6.1-24.1.zip

Kindly suggest.

Thanks,  
Shikha

---

<div class="post-metadata">

**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\
**Post date:** [March 12, 2019, 6:45am UTC](https://discuss.elastic.co/t/elastic-cluster-with-hadoop/171860/3 "2019-03-12T06:45:49Z")

</div>

Hi there,

[Kerberos is available](https://www.elastic.co/subscriptions) given the right license in the Elastic stack and our documentation is [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-kerberos-realm.html).

Searchguard is a 3rd party plugin, and you decide to use that you will probably get help quicker opening an issue at their relevant github repository

---

<div class="post-metadata">

**Author:** ![shikhasingh](https://avatars.discourse-cdn.com/v4/letter/s/35a633/32.png) [@shikhasingh](https://discuss.elastic.co/u/shikhasingh)\
**Post date:** [March 12, 2019, 12:24pm UTC](https://discuss.elastic.co/t/elastic-cluster-with-hadoop/171860/4 "2019-03-12T12:24:23Z")

</div>

Hi,

Thanks for the reply, We are using open source free version of ES, I have gone through many links and found that Security feature is not available for free version.

Please correct me if i am wrong.  
Our Hadoop cluster is kerberised and if we need to integrate ES with it we need to kerberised our ES cluster as well.

Please let me know if we should go with only licensed version of ES.

Thanks,  
Shikha

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 9, 2019, 12:24pm UTC](https://discuss.elastic.co/t/elastic-cluster-with-hadoop/171860/5 "2019-04-09T12:24:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.