hi folks,
I facing following error while load data into elasticsearch using hive
query.
ERROR:-
14/08/30 02:05:04 INFO log.PerfLogger:
14/08/30 02:05:04 INFO ql.Driver: Starting command: CREATE EXTERNAL TABLE
eslogs (time STRING, extension STRING, clientip STRING, request STRING,
response INT, agent STRING)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' =
'http://192.168.186.1','es.port'
= '9200')
14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:
http://sandbox.hortonworks.com:8188/ws/v1/timeline/
14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook
14/08/30 02:05:04 INFO log.PerfLogger:
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG
method=PreHook.org.apache.hadoop.hive.ql.hooks.ATSHook start=1409389504583
end=1409389504583 duration=0 from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG method=TimeToSubmit
start=1409389502353 end=1409389504583 duration=2230
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 INFO log.PerfLogger:
14/08/30 02:05:04 INFO log.PerfLogger:
14/08/30 02:05:04 INFO exec.DDLTask: Use StorageHandler-supplied
org.elasticsearch.hadoop.hive.EsSerDe for table eslogs
14/08/30 02:05:04 ERROR exec.DDLTask: java.lang.IllegalAccessError:
org/elasticsearch/hadoop/hive/HiveUtils
at org.elasticsearch.hadoop.hive.EsSerDe.initialize(EsSerDe.java:80)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:339)
at
org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:283)
at
org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:276)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:626)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:593)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4194)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:281)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:349)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
14/08/30 02:05:04 INFO impl.TimelineClientImpl: Timeline service address:
http://sandbox.hortonworks.com:8188/ws/v1/timeline/
14/08/30 02:05:04 INFO hooks.ATSHook: Created ATS Hook
14/08/30 02:05:04 INFO log.PerfLogger:
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG
method=FailureHook.org.apache.hadoop.hive.ql.hooks.ATSHook
start=1409389504698 end=1409389504698 duration=0
from=org.apache.hadoop.hive.ql.Driver>
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
org/elasticsearch/hadoop/hive/HiveUtils
14/08/30 02:05:04 ERROR ql.Driver: FAILED: Execution Error, return code 1
from org.apache.hadoop.hive.ql.exec.DDLTask.
org/elasticsearch/hadoop/hive/HiveUtils
14/08/30 02:05:04 INFO log.PerfLogger: </PERFLOG method=Driver.execute
start=1409389504462 end=1409389504700 duration=238
from=org.apache.hadoop.hive.ql.Driver>
14/08/30 02:05:04 ERROR beeswax.BeeswaxServiceImpl: Exception while
processing query
BeeswaxException(message:Driver returned: 1. Errors: OK
converting to local hdfs://
sandbox.hortonworks.com:8020/user/hue/elasticsearch-hadoop-2.1.0.Beta1.jar
Added
/tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841_resources/elasticsearch-hadoop-2.1.0.Beta1.jar
to class path
Added resource:
/tmp/a1c5d5b9-3aa7-41f8-8e93-73f8277a9841_resources/elasticsearch-hadoop-2.1.0.Beta1.jar
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
org/elasticsearch/hadoop/hive/HiveUtils
, log_context:d0960413-bfb9-4bdf-b615-26ad7821587b,
handle:QueryHandle(id:d0960413-bfb9-4bdf-b615-26ad7821587b,
log_context:d0960413-bfb9-4bdf-b615-26ad7821587b), SQLState: )
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:356)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:614)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:603)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1537)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:603)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
My setup:-
- Hortonworks Sandbox VM (running on vmware player):-
Hortonworks_Sandbox_2.1 - elasticsearch up & running :- Elasticsearch version 1.3.1
- elasticsearch-hadoop jars :- elasticsearch-hadoop-2.1.0.Beta1,
elasticsearch-hadoop-1.3.0.M1
I am following this url :-
https://github.com/hortonworks/hadoop-tutorials/blob/master/Community/T07_Elasticsearch_Hadoop_Integration.md
Hive Queries:-
Create table in hive and load data from zip file:-
CREATE TABLE apachelog (type STRING, time STRING, ext STRING, ip STRING,
req STRING, res INT, bytes INT, phpmem INT, agent STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';
LOAD DATA INPATH '/user/hue/apache/apache.log' OVERWRITE INTO TABLE
apachelog;
Load data into easlticsearch query:-
CREATE EXTERNAL TABLE eslogs (time STRING, extension STRING, clientip
STRING, request STRING, response INT, agent STRING)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'demo/hive','es.nodes' =
'http://192.168.186.1','es.port'
= '9200');
INSERT OVERWRITE TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res,
s.agent FROM apachelog s;
Kindly suggest me where I am doing wrong and what is the meaning of this
error and why it is coming?
Regrads
Mohit Kumar Yadav
(MCA/BBA)
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAK6eDKeY0%2B5r_WTh8mpFJiXJOMC2P__G254XZQDsXmJczHMtMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.