# ElasticSearch-Hadoop:Loading data into Elasticsearch through hive querl showing DDLTask error

**URL:** <https://discuss.elastic.co/t/elasticsearch-hadoop-loading-data-into-elasticsearch-through-hive-querl-showing-ddltask-error/19642>\
**Category:** Elasticsearch\
**Created:** [September 5, 2014, 12:39pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-loading-data-into-elasticsearch-through-hive-querl-showing-ddltask-error/19642 "2014-09-05T12:39:02Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohit\_Kumar](https://avatars.discourse-cdn.com/v4/letter/m/c89c15/32.png) [@mohit\_Kumar](https://discuss.elastic.co/u/mohit_Kumar)\
**Post date:** [September 5, 2014, 12:39pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-loading-data-into-elasticsearch-through-hive-querl-showing-ddltask-error/19642/1 "2014-09-05T12:39:02Z")

</div>

hi folks,  
I facing following error while load data into elasticsearch using hive  
query.  
ERROR:-  
14/08/30 02:05:04 INFO log.PerfLogger:   
14/08/30 02:05:04 INFO ql.Driver: Starting command: CREATE EXTERNAL TABLE  
eslogs (time STRING, extension STRING, clientip STRING, request STRING,  
response INT, agent STRING)  
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' =  
'[http://192.168.186.1](http://192.168.186.1)','es.port'  
= '9200')  
14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:  
[http://sandbox.hortonworks.com:8188/ws/v1/timeline/](http://sandbox.hortonworks.com:8188/ws/v1/timeline/)  
14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook  
14/08/30 02:05:04 INFO log.PerfLogger:   
14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG  
method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1409389504583  
end=1409389504583 duration=0 from=org.apache.hadoop.hive.ql.Driver\>  
14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=TimeToSubmit  
start=1409389502353 end=1409389504583 duration=2230  
from=org.apache.hadoop.hive.ql.Driver\>  
14/08/30 02:05:04 INFO log.PerfLogger:   
14/08/30 02:05:04 INFO log.PerfLogger:   
14/08/30 02:05:04 INFO exec.DDLTask: Use StorageHandler-supplied  
org.elasticsearch.hadoop.hive.EsSerDe for table eslogs  
14/08/30 02:05:04 ERROR exec.DDLTask: java.lang.IllegalAccessError:  
org/elasticsearch/hadoop/hive/HiveUtils  
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:80)  
at  
org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339)  
at  
org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:283)  
at  
org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:276)  
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:626)  
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:593)  
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194)  
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281)  
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)  
at  
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)  
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504)  
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271)  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:349)  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)  
at java.security.AccessController.doPrivileged(Native Method)  
at javax.security.auth.Subject.doAs(Subject.java:356)  
at  
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)  
at  
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
at java.util.concurrent.FutureTask.run(FutureTask.java:262)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)

14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:  
[http://sandbox.hortonworks.com:8188/ws/v1/timeline/](http://sandbox.hortonworks.com:8188/ws/v1/timeline/)  
14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook  
14/08/30 02:05:04 INFO log.PerfLogger:   
14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG  
method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook  
start=1409389504698 end=1409389504698 duration=0  
from=org.apache.hadoop.hive.ql.Driver\>  
FAILED: Execution Error, return code 1 from  
org.apache.hadoop.hive.ql.exec.DDLTask.  
org/elasticsearch/hadoop/hive/HiveUtils  
14/08/30 02:05:04 ERROR ql.Driver: FAILED: Execution Error, return code 1  
from org.apache.hadoop.hive.ql.exec.DDLTask.  
org/elasticsearch/hadoop/hive/HiveUtils  
14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=Driver.execute  
start=1409389504462 end=1409389504700 duration=238  
from=org.apache.hadoop.hive.ql.Driver\>  
14/08/30 02:05:04 ERROR beeswax.BeeswaxServiceImpl: Exception while  
processing query  
BeeswaxException(message:Driver returned: 1. Errors: OK  
converting to local hdfs://  
[sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar](http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar)  
Added  
/tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841\_resources/elasticsearch-hadoop-2.1.0.Beta1.jar  
to class path  
Added resource:  
/tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841\_resources/elasticsearch-hadoop-2.1.0.Beta1.jar  
FAILED: Execution Error, return code 1 from  
org.apache.hadoop.hive.ql.exec.DDLTask.  
org/elasticsearch/hadoop/hive/HiveUtils  
, log\_context:d0960413-bfb9-4bdf-b615-26ad7821587b,  
handle:QueryHandle(id:d0960413-bfb9-4bdf-b615-26ad7821587b,  
log\_context:d0960413-bfb9-4bdf-b615-26ad7821587b), SQLState: )  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356)  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)  
at java.security.AccessController.doPrivileged(Native Method)  
at javax.security.auth.Subject.doAs(Subject.java:356)  
at  
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)  
at  
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)  
at  
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
at java.util.concurrent.FutureTask.run(FutureTask.java:262)  
at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)

My setup:-

1. Hortonworks Sandbox VM (running on vmware player):-  
Hortonworks\_Sandbox\_2.1
2. elasticsearch up & running :- Elasticsearch version 1.3.1
3. elasticsearch-hadoop jars :- elasticsearch-hadoop-2.1.0.Beta1,  
elasticsearch-hadoop-1.3.0.M1

I am following this url :-  
[https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07\_Elasticsearch\_Hadoop\_Integration.md](https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07_Elasticsearch_Hadoop_Integration.md)

Hive Queries:-  
Create table in hive and load data from zip file:-

CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING,  
req STRING, res INT, bytes INT, phpmem INT, agent STRING)  
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';  
LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE  
apachelog;

Load data into easlticsearch query:-

CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip  
STRING, request STRING, response INT, agent STRING)  
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' =  
'[http://192.168.186.1](http://192.168.186.1)','es.port'  
= '9200');

INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res,  
s.agent FROM apachelog s;

Kindly suggest me where I am doing wrong and what is the meaning of this  
error and why it is coming?

Regrads  
Mohit Kumar Yadav  
(MCA/BBA)

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r\_WTh8mpFJiXJOMC2P\_\_G254XZQDsXmJczHMtMQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [September 5, 2014, 12:43pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-loading-data-into-elasticsearch-through-hive-querl-showing-ddltask-error/19642/2 "2014-09-05T12:43:17Z")

</div>

It looks like there's a classpath issue (notice the HiveUtils error there). Most likely because you have two versions  
of es-hadoop in your classpath (2.1.0.Beta1 and 1.3.0.M1).

Use only one - I suggest 2.1.0.Beta1.

Cheers,

On 9/5/14 3:39 PM, Mohit Kumar Yadav wrote:

> hi folks,  
> I facing following error while load data into elasticsearch using hive query.  
> ERROR:-  
> 14/08/30 02:05:04 INFO log.PerfLogger:   
> 14/08/30 02:05:04 INFO ql.Driver: Starting command: CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING,  
> clientip STRING, request STRING, response INT, agent STRING)  
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
> TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = '[http://192.168.186.1](http://192.168.186.1)','es.port' = '9200')  
> 14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:  
> [http://sandbox.hortonworks.com:8188/ws/v1/timeline/](http://sandbox.hortonworks.com:8188/ws/v1/timeline/)  
> 14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook  
> 14/08/30 02:05:04 INFO log.PerfLogger:   
> 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook  
> start=1409389504583 end=1409389504583 duration=0 from=org.apache.hadoop.hive.ql.Driver\>  
> 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=TimeToSubmit start=1409389502353 end=1409389504583 duration=2230  
> from=org.apache.hadoop.hive.ql.Driver\>  
> 14/08/30 02:05:04 INFO log.PerfLogger:   
> 14/08/30 02:05:04 INFO log.PerfLogger:   
> 14/08/30 02:05:04 INFO exec.DDLTask: Use StorageHandler-supplied org.elasticsearch.hadoop.hive.EsSerDe for table eslogs  
> 14/08/30 02:05:04 ERROR exec.DDLTask: java.lang.IllegalAccessError: org/elasticsearch/hadoop/hive/HiveUtils  
> at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:80)  
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339)  
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:283)  
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:276)  
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:626)  
> at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:593)  
> at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194)  
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281)  
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)  
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)  
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504)  
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271)  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:349)  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)  
> at java.security.AccessController.doPrivileged(Native Method)  
> at javax.security.auth.Subject.doAs(Subject.java:356)  
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)  
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)  
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:744)
> 
> 14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:  
> [http://sandbox.hortonworks.com:8188/ws/v1/timeline/](http://sandbox.hortonworks.com:8188/ws/v1/timeline/)  
> 14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook  
> 14/08/30 02:05:04 INFO log.PerfLogger:   
> 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook  
> start=1409389504698 end=1409389504698 duration=0 from=org.apache.hadoop.hive.ql.Driver\>  
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/elasticsearch/hadoop/hive/HiveUtils  
> 14/08/30 02:05:04 ERROR ql.Driver: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.  
> org/elasticsearch/hadoop/hive/HiveUtils  
> 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=Driver.execute start=1409389504462 end=1409389504700  
> duration=238 from=org.apache.hadoop.hive.ql.Driver\>  
> 14/08/30 02:05:04 ERROR beeswax.BeeswaxServiceImpl: Exception while processing query  
> BeeswaxException(message:Driver returned: 1. Errors: OK  
> converting to local hdfs://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar  
> [http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar](http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar)  
> Added /tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841\_resources/elasticsearch-hadoop-2.1.0.Beta1.jar to class path  
> Added resource: /tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841\_resources/elasticsearch-hadoop-2.1.0.Beta1.jar  
> FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org/elasticsearch/hadoop/hive/HiveUtils  
> , log\_context:d0960413-bfb9-4bdf-b615-26ad7821587b, handle:QueryHandle(id:d0960413-bfb9-4bdf-b615-26ad7821587b,  
> log\_context:d0960413-bfb9-4bdf-b615-26ad7821587b), SQLState: )  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356)  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)  
> at java.security.AccessController.doPrivileged(Native Method)  
> at javax.security.auth.Subject.doAs(Subject.java:356)  
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)  
> at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)  
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)  
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> at java.lang.Thread.run(Thread.java:744)
> 
> My setup:-
> 
> 1. Hortonworks Sandbox VM (running on vmware player):- Hortonworks\_Sandbox\_2.1
> 2. elasticsearch up & running :- Elasticsearch version 1.3.1
> 3. elasticsearch-hadoop jars :- elasticsearch-hadoop-2.1.0.Beta1,elasticsearch-hadoop-1.3.0.M1
> 
> I am following this url :-  
> [https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07\_Elasticsearch\_Hadoop\_Integration.md](https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07_Elasticsearch_Hadoop_Integration.md)
> 
> Hive Queries:-  
> Create table in hive and load data from zip file:-
> 
> CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING, req STRING, res INT, bytes INT, phpmem INT,  
> agent STRING)  
> ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';  
> LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE apachelog;
> 
> Load data into easlticsearch query:-
> 
> CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip STRING, request STRING, response INT, agent STRING)  
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
> TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = '[http://192.168.186.1](http://192.168.186.1)','es.port' = '9200');
> 
> INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res, s.agent FROM apachelog s;
> 
> Kindly suggest me where I am doing wrong and what is the meaning of this error and why it is coming?
> 
> Regrads  
> Mohit Kumar Yadav  
> (MCA/BBA)
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r\_WTh8mpFJiXJOMC2P\_\_G254XZQDsXmJczHMtMQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com)  
> [https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r\_WTh8mpFJiXJOMC2P\_\_G254XZQDsXmJczHMtMQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com?utm_medium=email&utm_source=footer).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
Costin

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/5409AFE5.9010904%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/5409AFE5.9010904%40gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![mohit\_Kumar](https://avatars.discourse-cdn.com/v4/letter/m/c89c15/32.png) [@mohit\_Kumar](https://discuss.elastic.co/u/mohit_Kumar)\
**Post date:** [September 8, 2014, 12:50pm UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-loading-data-into-elasticsearch-through-hive-querl-showing-ddltask-error/19642/3 "2014-09-08T12:50:07Z")

</div>

Thanks Costin for replying,

That works for me but now I am facing with another issue.

data are not loading into elasticsearch. whenever I fires insertion query  
it shows following error.

Query ID = hue\_20140908052626\_c15ea214-c656-47b1-9132-49d3c407b8f8  
Total jobs = 1  
Launching Job 1 out of 1  
Number of reduce tasks is set to 0 since there's no reduce operator  
Starting Job = job\_1410175571220\_0003, Tracking URL = [http://sandbox.hortonworks.com:8088/proxy/application\_1410175571220\_0003/](http://sandbox.hortonworks.com:8088/proxy/application_1410175571220_0003/)  
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job\_1410175571220\_0003  
Hadoop job information for Stage-0: number of mappers: 1; number of reducers: 0  
2014-09-08 05:26:57,048 Stage-0 map = 0%, reduce = 0%  
2014-09-08 05:27:29,019 Stage-0 map = 100%, reduce = 0%  
Ended Job = job\_1410175571220\_0003 with errors  
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask  
MapReduce Jobs Launched:  
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL  
Total MapReduce CPU Time Spent: 0 msec  
, log\_context:d3f5bc11-575d-4dd5-91a4-c6d9c0032c25, handle:QueryHandle(id:d3f5bc11-575d-4dd5-91a4-c6d9c0032c25, log\_context:d3f5bc11-575d-4dd5-91a4-c6d9c0032c25), SQLState: )  
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356)  
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)  
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)  
at java.security.AccessController.doPrivileged(Native Method)  
at javax.security.auth.Subject.doAs(Subject.java:356)  
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)  
at com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)  
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)  
at java.util.concurrent.FutureTask.run(FutureTask.java:262)  
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
at java.lang.Thread.run(Thread.java:744)

Hive Queries:-Create table in hive and load data from zip file:-CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING, req STRING, res INT, bytes INT, phpmem INT, agent STRING)ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE apachelog;  
Load data into easlticsearch query:-CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip STRING, request STRING, response INT, agent STRING)STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = '[http://192.168.186.1](http://192.168.186.1)','es.port' = '9200');  
INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res, s.agent FROM apachelog s;  
please suggest me what should i do to remove this error.

thanks in advance

On Friday, September 5, 2014 6:13:58 PM UTC+5:30, Costin Leau wrote:

> It looks like there's a classpath issue (notice the HiveUtils error  
> there). Most likely because you have two versions  
> of es-hadoop in your classpath (2.1.0.Beta1 and 1.3.0.M1).
> 
> Use only one - I suggest 2.1.0.Beta1.
> 
> Cheers,
> 
> On 9/5/14 3:39 PM, Mohit Kumar Yadav wrote:
> 
> > hi folks,  
> > I facing following error while load data into elasticsearch using hive  
> > query.  
> > ERROR:-  
> > 14/08/30 02:05:04 INFO log.PerfLogger:   
> > 14/08/30 02:05:04 INFO ql.Driver: Starting command: CREATE EXTERNAL  
> > TABLE eslogs (time STRING, extension STRING,  
> > clientip STRING, request STRING, response INT, agent STRING)  
> > STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
> > TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = '  
> > [http://192.168.186.1](http://192.168.186.1)','es.port' = '9200')  
> > 14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service  
> > address:  
> > [http://sandbox.hortonworks.com:8188/ws/v1/timeline/](http://sandbox.hortonworks.com:8188/ws/v1/timeline/)  
> > 14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook  
> > 14/08/30 02:05:04 INFO log.PerfLogger:   
> > 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG  
> > method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook  
> > start=1409389504583 end=1409389504583 duration=0  
> > from=org.apache.hadoop.hive.ql.Driver\>  
> > 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=TimeToSubmit  
> > start=1409389502353 end=1409389504583 duration=2230  
> > from=org.apache.hadoop.hive.ql.Driver\>  
> > 14/08/30 02:05:04 INFO log.PerfLogger:   
> > 14/08/30 02:05:04 INFO log.PerfLogger:   
> > 14/08/30 02:05:04 INFO exec.DDLTask: Use StorageHandler-supplied  
> > org.elasticsearch.hadoop.hive.EsSerDe for table eslogs  
> > 14/08/30 02:05:04 ERROR exec.DDLTask: java.lang.IllegalAccessError:  
> > org/elasticsearch/hadoop/hive/HiveUtils  
> > at  
> > org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:80)  
> > at  
> > org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339)
> 
> > ```
> > at 
> > 
> > ```
> 
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:283)
> 
> > ```
> > at 
> > 
> > ```
> 
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:276)
> 
> > ```
> > at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:626) 
> > at 
> > 
> > ```
> 
> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:593)
> 
> > ```
> > at 
> > 
> > ```
> 
> org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194)
> 
> > ```
> > at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281) 
> > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153) 
> > at 
> > 
> > ```
> 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
> 
> > ```
> > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504) 
> > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271) 
> > at 
> > 
> > ```
> 
> com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:349)
> 
> > ```
> > at 
> > 
> > ```
> 
> com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)
> 
> > ```
> > at 
> > 
> > ```
> 
> com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)
> 
> > ```
> > at java.security.AccessController.doPrivileged(Native Method) 
> > at javax.security.auth.Subject.doAs(Subject.java:356) 
> > at 
> > 
> > ```
> 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)
> 
> > ```
> > at 
> > 
> > ```
> 
> com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)
> 
> > ```
> > at 
> > 
> > ```
> 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 
> > ```
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> > at 
> > 
> > ```
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 
> > ```
> > at 
> > 
> > ```
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 
> > ```
> > at java.lang.Thread.run(Thread.java:744) 
> > 
> > ```
> > 
> > 14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service  
> > address:  
> > [http://sandbox.hortonworks.com:8188/ws/v1/timeline/](http://sandbox.hortonworks.com:8188/ws/v1/timeline/)  
> > 14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook  
> > 14/08/30 02:05:04 INFO log.PerfLogger:   
> > 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG  
> > method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook  
> > start=1409389504698 end=1409389504698 duration=0  
> > from=org.apache.hadoop.hive.ql.Driver\>  
> > FAILED: Execution Error, return code 1 from  
> > org.apache.hadoop.hive.ql.exec.DDLTask.  
> > org/elasticsearch/hadoop/hive/HiveUtils  
> > 14/08/30 02:05:04 ERROR ql.Driver: FAILED: Execution Error, return code  
> > 1 from org.apache.hadoop.hive.ql.exec.DDLTask.  
> > org/elasticsearch/hadoop/hive/HiveUtils  
> > 14/08/30 02:05:04 INFO log.PerfLogger: \</PERFLOG method=Driver.execute  
> > start=1409389504462 end=1409389504700  
> > duration=238 from=org.apache.hadoop.hive.ql.Driver\>  
> > 14/08/30 02:05:04 ERROR beeswax.BeeswaxServiceImpl: Exception while  
> > processing query  
> > BeeswaxException(message:Driver returned: 1. Errors: OK  
> > converting to local hdfs://  
> > [sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar](http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar)  
> > \<  
> > [http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar](http://sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar)\>
> 
> > Added  
> > /tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841\_resources/elasticsearch-hadoop-2.1.0.Beta1.jar  
> > to class path  
> > Added resource:  
> > /tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841\_resources/elasticsearch-hadoop-2.1.0.Beta1.jar
> 
> > FAILED: Execution Error, return code 1 from  
> > org.apache.hadoop.hive.ql.exec.DDLTask.  
> > org/elasticsearch/hadoop/hive/HiveUtils  
> > , log\_context:d0960413-bfb9-4bdf-b615-26ad7821587b,  
> > handle:QueryHandle(id:d0960413-bfb9-4bdf-b615-26ad7821587b,  
> > log\_context:d0960413-bfb9-4bdf-b615-26ad7821587b), SQLState: )  
> > at  
> > com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356)
> 
> > ```
> > at 
> > 
> > ```
> 
> com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)
> 
> > ```
> > at 
> > 
> > ```
> 
> com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)
> 
> > ```
> > at java.security.AccessController.doPrivileged(Native Method) 
> > at javax.security.auth.Subject.doAs(Subject.java:356) 
> > at 
> > 
> > ```
> 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)
> 
> > ```
> > at 
> > 
> > ```
> 
> com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)
> 
> > ```
> > at 
> > 
> > ```
> 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 
> > ```
> > at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
> > at 
> > 
> > ```
> 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 
> > ```
> > at 
> > 
> > ```
> 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 
> > ```
> > at java.lang.Thread.run(Thread.java:744) 
> > 
> > ```
> > 
> > My setup:-
> > 
> > 1. Hortonworks Sandbox VM (running on vmware player):-  
> > Hortonworks\_Sandbox\_2.1
> > 2. elasticsearch up & running :- Elasticsearch version 1.3.1
> > 3. elasticsearch-hadoop jars :-  
> > elasticsearch-hadoop-2.1.0.Beta1,elasticsearch-hadoop-1.3.0.M1
> > 
> > I am following this url :-
> 
> [https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07\_Elasticsearch\_Hadoop\_Integration.md](https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07_Elasticsearch_Hadoop_Integration.md)
> 
> > Hive Queries:-  
> > Create table in hive and load data from zip file:-
> > 
> > CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING,  
> > req STRING, res INT, bytes INT, phpmem INT,  
> > agent STRING)  
> > ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';  
> > LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE  
> > apachelog;
> > 
> > Load data into easlticsearch query:-
> > 
> > CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip  
> > STRING, request STRING, response INT, agent STRING)  
> > STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
> > TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' = '  
> > [http://192.168.186.1](http://192.168.186.1)','es.port' = '9200');
> > 
> > INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res,  
> > s.agent FROM apachelog s;
> > 
> > Kindly suggest me where I am doing wrong and what is the meaning of this  
> > error and why it is coming?
> > 
> > Regrads  
> > Mohit Kumar Yadav  
> > (MCA/BBA)
> > 
> > --  
> > You received this message because you are subscribed to the Google  
> > Groups "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send  
> > an email to  
> > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\> \<mailto:  
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \<javascript:\>\>.  
> > To view this discussion on the web visit
> 
> [https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r\_WTh8mpFJiXJOMC2P\_\_G254XZQDsXmJczHMtMQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com)
> 
> > \<  
> > [https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r\_WTh8mpFJiXJOMC2P\_\_G254XZQDsXmJczHMtMQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com?utm_medium=email&utm_source=footer)\>.
> 
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> Costin

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/440d87a5-47f4-4ff9-944a-dd01f5b367eb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/440d87a5-47f4-4ff9-944a-dd01f5b367eb%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:03am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-loading-data-into-elasticsearch-through-hive-querl-showing-ddltask-error/19642/4 "2017-07-06T01:03:56Z")

</div>