Hi,
Using hadoop and elasticsearch on cloud
created external table but getting some errors while inserting the data into the external table
set hive.execution.engine=mr
hive> insert overwrite table vgsale5es select * from vgsale5;
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = saivarunakuraju_20190829180559_6bd94dfb-f0fa-4c58-a4f6-963a3e10deb3
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1567078294077_0011, Tracking URL = http://hadoopcluster-m:8088/proxy/application_1567078294077_0011/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1567078294077_0011
Hadoop job information for Stage-3: number of mappers: 1; number of reducers: 0
2019-08-29 18:06:13,379 Stage-3 map = 0%, reduce = 0%
2019-08-29 18:07:13,827 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 2.02 sec
2019-08-29 18:08:14,062 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 2.02 sec
2019-08-29 18:09:14,304 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 1.57 sec
2019-08-29 18:10:14,527 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 1.57 sec
2019-08-29 18:11:14,669 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 1.97 sec
2019-08-29 18:12:14,784 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 1.97 sec
2019-08-29 18:13:14,840 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 1.97 sec
2019-08-29 18:14:14,973 Stage-3 map = 0%, reduce = 0%, Cumulative CPU 1.97 sec
2019-08-29 18:14:48,571 Stage-3 map = 100%, reduce = 0%
MapReduce Total cumulative CPU time: 1 seconds 970 msec
Ended Job = job_1567078294077_0011 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1567078294077_0011_m_000000 (and more) from job job_1567078294077_0011
Task with the most failures(4):
-----
Task ID:
task_1567078294077_0011_m_000000
URL:
http://hadoopcluster-m:8088/taskdetails.jsp?jobid=job_1567078294077_0011&tipid=task_1567078294077_0011_m_000000
-----
Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"rank":"rank","name":"name","platform":"platform","year":"year","genre":"genre","publisher":"publisher","na_sales":"na_sales","eu_sales":"eu_ales","jp_sales":"jp_sales","other_sales":"other_sales","global_sales":"global_sales"}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:169)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"rank":"rank","name":"name","platform":"platform","year":"year","genre":"genre","publisher":"publisher","na_sales":"na_sales","eu_sales":"eu_ales","jp_sales":"jp_sales","other_sales":"other_sales","global_sales":"global_sales"}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:562)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:160)
... 8 more
Caused by: org.elasticsearch.hadoop.rest.EsHadoopNoNodesLeftException: Connection error (check network and/or proxy settings)- all nodes failed; tried [[10.168.0.3:9200]]
at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:152)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:424)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:388)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:392)
at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:168)
at org.elasticsearch.hadoop.rest.request.GetAliasesRequestBuilder.execute(GetAliasesRequestBuilder.java:68)
at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:622)
at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:175)
at org.elasticsearch.hadoop.hive.EsHiveOutputFormat$EsHiveRecordWriter.write(EsHiveOutputFormat.java:59)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:762)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:897)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:130)
at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:148)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:547)
... 9 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-3: Map: 1 Cumulative CPU: 1.97 sec HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 1 seconds 970 msec
thanks