Hive write data to elastic search

Hi all,

I am trying to write data to elastic search from hive and whenever I try I
get this error:

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No resource
['es.resource'] (index/query/location) specified

The script I am running looks like this:

USE pl_10;

ADD jar /home/hdfs/sql-tests/schema/elasticsearch-hadoop-2.0.0.jar;

CREATE EXTERNAL TABLE IF NOT EXISTS REGIONES (

    R_REGIONKEY  INT,

R_NAME       STRING,

R_COMMENT    STRING

) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'

STORED by 'org.elasticsearch.hadoop.hive.EsStorageHandler'

TBLPROPERTIES('es.resource'='radio/artists',

'es.nodes'='elastic-1');

INSERT OVERWRITE TABLE REGIONES select R_REGIONKEY, R_NAME, R_COMMENT from
REGION;

Any help would be much appreciated

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/daadb18d-f086-459d-8bee-1e9060d96830%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

HI David,

did you find any fix for this issue, I am also facing the same problem.

Thanks

  • Atul

On Tuesday, July 8, 2014 8:37:09 AM UTC-7, David Zabner wrote:

Hi all,

I am trying to write data to Elasticsearch from hive and whenever I try I
get this error:

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No resource
['es.resource'] (index/query/location) specified

The script I am running looks like this:

USE pl_10;

ADD jar /home/hdfs/sql-tests/schema/elasticsearch-hadoop-2.0.0.jar;

CREATE EXTERNAL TABLE IF NOT EXISTS REGIONES (

    R_REGIONKEY  INT,

R_NAME       STRING,

R_COMMENT    STRING

) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'

STORED by 'org.elasticsearch.hadoop.hive.EsStorageHandler'

TBLPROPERTIES('es.resource'='radio/artists',

'es.nodes'='elastic-1');

INSERT OVERWRITE TABLE REGIONES select R_REGIONKEY, R_NAME, R_COMMENT from
REGION;

Any help would be much appreciated

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Atul,

What does your Hive script looks like? What version of hive and es-hadoop are you using? Can you post them
along with the stacktrace on Gist or Pastebin [1]?
The exception message is pretty straight-forward - either 'es.resource' is missing or the resource
type is incorrectly specified [2].

Cheers,

[1] Elasticsearch Platform — Find real-time answers at scale | Elastic
[2] Elasticsearch Platform — Find real-time answers at scale | Elastic

On 11/27/14 8:57 PM, Atul Paldhikar wrote:

HI David,

did you find any fix for this issue, I am also facing the same problem.

Thanks

  • Atul

On Tuesday, July 8, 2014 8:37:09 AM UTC-7, David Zabner wrote:

Hi all,

I am trying to write data to elastic search from hive and whenever I try I get this error:

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No resource ['es.resource'] (index/query/location) specified


The script I am running looks like this:

USE pl_10;

ADD jar /home/hdfs/sql-tests/schema/elasticsearch-hadoop-2.0.0.jar;

CREATE EXTERNAL TABLE IF NOT EXISTS REGIONES (

         R_REGIONKEY  INT,

     R_NAME       STRING,

     R_COMMENT    STRING

) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'

STORED by'org.elasticsearch.hadoop.hive.EsStorageHandler'

TBLPROPERTIES('es.resource'='radio/artists',

'es.nodes'='elastic-1');

INSERT OVERWRITE TABLE REGIONES select R_REGIONKEY, R_NAME, R_COMMENT from REGION;


Any help would be much appreciated

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5477ACC3.9050407%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Hi Costin,

actually I think I figured out the issue, my script had a typo (resources
instead of resource)

create external table ex_address (name String, st_no INT, st_name string,
city string, state string, zip INT) stored by
'org.elasticsearch.hadoop.hive.EsStorageHandler'
tblproperties('es.resources' = 'employee/address');

However, for some reason I am back to square one with the original problem
mentioned in another thread [Can't integrate Elasticsearch with Hive]. I
have started getting the old exception again when accessing the external
table, nothing really changed in the environment !

java.lang.IllegalAccessError: tried to access class
org.elasticsearch.hadoop.hive.HiveUtils from class
org.elasticsearch.hadoop.hive.EsSerDe

Thanks

  • Atul

On Thursday, November 27, 2014 2:59:49 PM UTC-8, Costin Leau wrote:

Hi Atul,

What does your Hive script looks like? What version of hive and es-hadoop
are you using? Can you post them
along with the stacktrace on Gist or Pastebin [1]?
The exception message is pretty straight-forward - either 'es.resource' is
missing or the resource
type is incorrectly specified [2].

Cheers,

[1]
Elasticsearch Platform — Find real-time answers at scale | Elastic
[2]
Elasticsearch Platform — Find real-time answers at scale | Elastic

On 11/27/14 8:57 PM, Atul Paldhikar wrote:

HI David,

did you find any fix for this issue, I am also facing the same problem.

Thanks

  • Atul

On Tuesday, July 8, 2014 8:37:09 AM UTC-7, David Zabner wrote:

Hi all, 

I am trying to write data to elastic search from hive and whenever I 

try I get this error:

org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No 

resource ['es.resource'] (index/query/location) specified

The script I am running looks like this: 

USE pl_10; 

ADD jar /home/hdfs/sql-tests/schema/elasticsearch-hadoop-2.0.0.jar; 

CREATE EXTERNAL TABLE IF NOT EXISTS REGIONES ( 

         R_REGIONKEY  INT, 

     R_NAME       STRING, 

     R_COMMENT    STRING 

) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 

STORED by'org.elasticsearch.hadoop.hive.EsStorageHandler' 

TBLPROPERTIES('es.resource'='radio/artists', 

'es.nodes'='elastic-1'); 

INSERT OVERWRITE TABLE REGIONES select R_REGIONKEY, R_NAME, 

R_COMMENT from REGION;

Any help would be much appreciated 

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:> <mailto:
elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com

<
https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/41b72358-82ca-4b21-bd59-b5b8f7dc2adf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Based on the other email that you sent, I assume this issue was fixed as well, right?

On 11/28/14 8:44 AM, Atul Paldhikar wrote:

Hi Costin,

actually I think I figured out the issue, my script had a typo (resources instead of resource)

create external table ex_address (name String, st_no INT, st_name string, city string, state string, zip INT) stored by
'org.elasticsearch.hadoop.hive.EsStorageHandler' tblproperties('es.resources' = 'employee/address');

However, for some reason I am back to square one with the original problem mentioned in another thread [Can't integrate
Elasticsearch with Hive]. I have started getting the old exception again when accessing the external table, nothing
really changed in the environment !

java.lang.IllegalAccessError: tried to access class org.elasticsearch.hadoop.hive.HiveUtils from class
org.elasticsearch.hadoop.hive.EsSerDe

Thanks

  • Atul

On Thursday, November 27, 2014 2:59:49 PM UTC-8, Costin Leau wrote:

Hi Atul,

What does your Hive script looks like? What version of hive and es-hadoop are you using? Can you post them
along with the stacktrace on Gist or Pastebin [1]?
The exception message is pretty straight-forward - either 'es.resource' is missing or the resource
type is incorrectly specified [2].

Cheers,

[1] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/troubleshooting.html>
[2] http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/hive.html#hive-configuration
<http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/hive.html#hive-configuration>

On 11/27/14 8:57 PM, Atul Paldhikar wrote:
> HI David,
>
> did you find any fix for this issue, I am also facing the same problem.
>
> Thanks
> - Atul
>
> On Tuesday, July 8, 2014 8:37:09 AM UTC-7, David Zabner wrote:
>
>     Hi all,
>
>     I am trying to write data to elastic search from hive and whenever I try I get this error:
>
>     org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No resource ['es.resource'] (index/query/location) specified
>
>
>     The script I am running looks like this:
>
>     USE pl_10;
>
>     ADD jar /home/hdfs/sql-tests/schema/elasticsearch-hadoop-2.0.0.jar;
>
>     CREATE EXTERNAL TABLE IF NOT EXISTS REGIONES (
>
>              R_REGIONKEY  INT,
>
>          R_NAME       STRING,
>
>          R_COMMENT    STRING
>
>     ) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
>
>     STORED by'org.elasticsearch.hadoop.hive.EsStorageHandler'
>
>     TBLPROPERTIES('es.resource'='radio/artists',
>
>     'es.nodes'='elastic-1');
>
>     INSERT OVERWRITE TABLE REGIONES select R_REGIONKEY, R_NAME, R_COMMENT from REGION;
>
>
>     Any help would be much appreciated
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
>elasticsearc...@googlegroups.com <javascript:> <mailto:elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
> To view this discussion on the web visit
>https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com>
> <https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <https://groups.google.com/d/optout>.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/41b72358-82ca-4b21-bd59-b5b8f7dc2adf%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/41b72358-82ca-4b21-bd59-b5b8f7dc2adf%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/547865F9.5050604%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

yes that's correct, the issue is resolved.

Thanks

  • Atul

On Friday, November 28, 2014 4:09:41 AM UTC-8, Costin Leau wrote:

Based on the other email that you sent, I assume this issue was fixed as
well, right?

On 11/28/14 8:44 AM, Atul Paldhikar wrote:

Hi Costin,

actually I think I figured out the issue, my script had a typo
(resources instead of resource)

create external table ex_address (name String, st_no INT, st_name
string, city string, state string, zip INT) stored by
'org.elasticsearch.hadoop.hive.EsStorageHandler'
tblproperties('es.resources' = 'employee/address');

However, for some reason I am back to square one with the original
problem mentioned in another thread [Can't integrate
Elasticsearch with Hive]. I have started getting the old exception again
when accessing the external table, nothing
really changed in the environment !

java.lang.IllegalAccessError: tried to access class
org.elasticsearch.hadoop.hive.HiveUtils from class
org.elasticsearch.hadoop.hive.EsSerDe

Thanks

  • Atul

On Thursday, November 27, 2014 2:59:49 PM UTC-8, Costin Leau wrote:

Hi Atul, 

What does your Hive script looks like? What version of hive and 

es-hadoop are you using? Can you post them

along with the stacktrace on Gist or Pastebin [1]? 
The exception message is pretty straight-forward - either 

'es.resource' is missing or the resource

type is incorrectly specified [2]. 

Cheers, 

[1] 

Elasticsearch Platform — Find real-time answers at scale | Elastic

<

Elasticsearch Platform — Find real-time answers at scale | Elastic>

[2] 

Elasticsearch Platform — Find real-time answers at scale | Elastic

<

Elasticsearch Platform — Find real-time answers at scale | Elastic>

On 11/27/14 8:57 PM, Atul Paldhikar wrote: 
> HI David, 
> 
> did you find any fix for this issue, I am also facing the same 

problem.

> 
> Thanks 
> - Atul 
> 
> On Tuesday, July 8, 2014 8:37:09 AM UTC-7, David Zabner wrote: 
> 
>     Hi all, 
> 
>     I am trying to write data to elastic search from hive and 

whenever I try I get this error:

> 
>     org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: No 

resource ['es.resource'] (index/query/location) specified

> 
> 
>     The script I am running looks like this: 
> 
>     USE pl_10; 
> 
>     ADD jar 

/home/hdfs/sql-tests/schema/elasticsearch-hadoop-2.0.0.jar;

> 
>     CREATE EXTERNAL TABLE IF NOT EXISTS REGIONES ( 
> 
>              R_REGIONKEY  INT, 
> 
>          R_NAME       STRING, 
> 
>          R_COMMENT    STRING 
> 
>     ) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' 
> 
>     STORED by'org.elasticsearch.hadoop.hive.EsStorageHandler' 
> 
>     TBLPROPERTIES('es.resource'='radio/artists', 
> 
>     'es.nodes'='elastic-1'); 
> 
>     INSERT OVERWRITE TABLE REGIONES select R_REGIONKEY, R_NAME, 

R_COMMENT from REGION;

> 
> 
>     Any help would be much appreciated 
> 
> -- 
> You received this message because you are subscribed to the Google 

Groups "elasticsearch" group.

> To unsubscribe from this group and stop receiving emails from it, 

send an email to

>elasticsearc...@googlegroups.com <javascript:> <mailto:

elasticsearch+unsubscribe@googlegroups.com <javascript:> <javascript:>>.

> To view this discussion on the web visit 
>

https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com

<

https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com>

> <

https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com?utm_medium=email&utm_source=footer

<

https://groups.google.com/d/msgid/elasticsearch/54cced44-4f87-4569-923f-30e406ddb79c%40googlegroups.com?utm_medium=email&utm_source=footer>>.

> For more options, visithttps://groups.google.com/d/optout <

https://groups.google.com/d/optout>.

-- 
Costin 

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to
elasticsearc...@googlegroups.com <javascript:> <mailto:
elasticsearch+unsubscribe@googlegroups.com <javascript:>>.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/41b72358-82ca-4b21-bd59-b5b8f7dc2adf%40googlegroups.com

<
https://groups.google.com/d/msgid/elasticsearch/41b72358-82ca-4b21-bd59-b5b8f7dc2adf%40googlegroups.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d0b84e2e-9a00-4ca5-87c9-43188dc3c8c7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.