# \[elasticsearch-hadoop problem\] run hive script for two table join. some error happened. Two tables also stored in elasticsearch
**URL:** https://discuss.elastic.co/t/elasticsearch-hadoop-problem-run-hive-script-for-two-table-join-some-error-happened-two-tables-also-stored-in-elasticsearch/16449
**Category:** Elasticsearch
**Created:** [March 19, 2014, 7:01am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-problem-run-hive-script-for-two-table-join-some-error-happened-two-tables-also-stored-in-elasticsearch/16449 "2014-03-19T07:01:00Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@11139](https://discuss.elastic.co/u/11139)
#### Post date: [March 19, 2014, 7:01am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-problem-run-hive-script-for-two-table-join-some-error-happened-two-tables-also-stored-in-elasticsearch/16449/1 "2014-03-19T07:01:00Z")
Dear all:
```
Recently I come across a strange problem. I want to use the
```
elasticsearch-1.0.0 as a backend storage for hive. I use the
elasticsearch-hadoop-1.3.0.M2 to create hive tables on elasticsearch. The
hive sql are as followings:
create external table supplier\_es (S\_SUPPKEY BIGINT, S\_NAME STRING,
S\_ADDRESS STRING, S\_NATIONKEY BIGINT, S\_PHONE STRING, S\_ACCTBAL DOUBLE,
S\_COMMENT STRING) stored by
'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource'='q9/supplier','es.index.auto.create'='true'，'es.nodes'
= 'localhost:9200');
create external table nation\_es (N\_NATIONKEY BIGINT, N\_NAME STRING,
N\_REGIONKEY BIGINT, N\_COMMENT STRING) stored by
'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource'='q9/nation','es.index.auto.create'='true','es.nodes'
= 'localhost:9200');
The table join operation is as followings:
select s\_suppkey, n\_name from supplier\_es s join nation\_es n on
n.n\_nationkey = s.s\_nationkey;
the error messages( I get from the log file):
2014-03-19 14:21:03,254 INFO [main]
org.apache.hadoop.conf.Configuration.deprecation: map.input.file is
deprecated. Instead, use mapreduce.map.input.file
2014-03-19 14:21:03,254 INFO [main]
org.apache.hadoop.hive.ql.exec.MapOperator:
fpath:hdfs://server-220.novalocal:8020/user/hive/warehouse/nation\_es
2014-03-19 14:21:03,268 INFO [main]
org.apache.hadoop.hive.ql.exec.MapOperator: getPathToAliases
2014-03-19 14:21:03,268 INFO [main]
org.apache.hadoop.hive.ql.exec.MapOperator: key :
hdfs://server-220.novalocal:8020/user/hive/warehouse/supplier\_es
2014-03-19 14:21:03,268 INFO [main]
org.apache.hadoop.hive.ql.exec.MapOperator: value[s]
2014-03-19 14:21:03,268 INFO [main]
org.apache.hadoop.hive.ql.exec.MapOperator: Adding alias s to work list for
file hdfs://server-220.novalocal:8020/user/hive/warehouse/supplier\_es
2014-03-19 14:21:03,270 ERROR [main]
org.apache.hadoop.hive.ql.exec.MapOperator: Configuration does not have any
alias for path:
hdfs://server-220.novalocal:8020/user/hive/warehouse/nation\_es
2014-03-19 14:21:03,285 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.lang.RuntimeException: Error in configuring
object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at
org.apache.hadoop.mapred.MapTask.runOldMapper\_aroundBody2(MapTask.java:434)
at org.apache.hadoop.mapred.MapTask$AjcClosure3.run(MapTask.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at
com.intel.bigdata.management.agent.HadoopTaskAspect.doPhaseCall(HadoopTaskAspect.java:166)
at
com.intel.bigdata.management.agent.HadoopTaskAspect.ajc$inlineAccessMethod$com\_intel\_bigdata\_management\_agent\_HadoopTaskAspect$com\_intel\_bigdata\_management\_agent\_HadoopTaskAspect$doPhaseCall(HadoopTaskAspect.java:1)
at
com.intel.bigdata.management.agent.HadoopTaskAspect.aroundMap(HadoopTaskAspect.java:38)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:411)
at org.apache.hadoop.mapred.MapTask.run\_aroundBody0(MapTask.java:343)
at org.apache.hadoop.mapred.MapTask$AjcClosure1.run(MapTask.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at
com.intel.bigdata.management.agent.HadoopTaskAspect.aroundTaskRun(HadoopTaskAspect.java:95)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1531)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 19 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 24 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 27 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:142)
... 32 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input
path are inconsistent
at
org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:419)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:110)
... 32 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration
and input path are inconsistent
at
org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:413)
... 33 more
I have try to figure out the problem, but I can't find out the reason. I
ask anyone for help. Thanks very much.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4bcaef49-7483-4777-a8a2-6af671a082af%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4bcaef49-7483-4777-a8a2-6af671a082af%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@costin](https://discuss.elastic.co/u/costin)
#### Post date: [March 19, 2014, 8:10am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-problem-run-hive-script-for-two-table-join-some-error-happened-two-tables-also-stored-in-elasticsearch/16449/2 "2014-03-19T08:10:08Z")
Hi,
The issue might be caused by the fact that M2 doesn't support different input and output indices for the same job; that
is to use ES both as input and output within the same job (which is essentially what you are doing with the select).
This has been fixed in master - can you try the latest nightly build or potentially build master yourself?
Cheers,
On 3/19/14 9:01 AM, 沈国权 wrote:
> Dear all:
>
> ```
> Recently I come across a strange problem. I want to use the elasticsearch-1.0.0 as a backend storage for hive.
>
> ```
>
> I use the elasticsearch-hadoop-1.3.0.M2 to create hive tables on elasticsearch. The hive sql are as followings:
>
> create external table supplier\_es (S\_SUPPKEY BIGINT, S\_NAME STRING, S\_ADDRESS STRING, S\_NATIONKEY BIGINT, S\_PHONE
> STRING, S\_ACCTBAL DOUBLE, S\_COMMENT STRING) stored by 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource'='q9/supplier','es.index.auto.create'='true'，'es.nodes' = 'localhost:9200');
>
> create external table nation\_es (N\_NATIONKEY BIGINT, N\_NAME STRING, N\_REGIONKEY BIGINT, N\_COMMENT STRING) stored by
> 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource'='q9/nation','es.index.auto.create'='true','es.nodes' = 'localhost:9200');
>
> The table join operation is as followings:
>
> select s\_suppkey, n\_name from supplier\_es s join nation\_es n on n.n\_nationkey = s.s\_nationkey;
>
> the error messages( I get from the log file):
>
> 2014-03-19 14:21:03,254 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: map.input.file is deprecated.
> Instead, use mapreduce.map.input.file
> 2014-03-19 14:21:03,254 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator:
> fpath:hdfs://server-220.novalocal:8020/user/hive/warehouse/nation\_es
> 2014-03-19 14:21:03,268 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: getPathToAliases
>
> 2014-03-19 14:21:03,268 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: key :
> hdfs://server-220.novalocal:8020/user/hive/warehouse/supplier\_es
> 2014-03-19 14:21:03,268 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: value[s]
> 2014-03-19 14:21:03,268 INFO [main] org.apache.hadoop.hive.ql.exec.MapOperator: Adding alias s to work list for file
> hdfs://server-220.novalocal:8020/user/hive/warehouse/supplier\_es
> 2014-03-19 14:21:03,270 ERROR [main] org.apache.hadoop.hive.ql.exec.MapOperator: Configuration does not have any alias
> for path: hdfs://server-220.novalocal:8020/user/hive/warehouse/nation\_es
> 2014-03-19 14:21:03,285 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child :
> java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> at org.apache.hadoop.mapred.MapTask.runOldMapper\_aroundBody2(MapTask.java:434)
> at org.apache.hadoop.mapred.MapTask$AjcClosure3.run(MapTask.java:1)
> at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> at com.intel.bigdata.management.agent.HadoopTaskAspect.doPhaseCall(HadoopTaskAspect.java:166)
> at
> com.intel.bigdata.management.agent.HadoopTaskAspect.ajc$inlineAccessMethod$com\_intel\_bigdata\_management\_agent\_HadoopTaskAspect$com\_intel\_bigdata\_management\_agent\_HadoopTaskAspect$doPhaseCall(HadoopTaskAspect.java:1)
> at com.intel.bigdata.management.agent.HadoopTaskAspect.aroundMap(HadoopTaskAspect.java:38)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:411)
> at org.apache.hadoop.mapred.MapTask.run\_aroundBody0(MapTask.java:343)
> at org.apache.hadoop.mapred.MapTask$AjcClosure1.run(MapTask.java:1)
> at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
> at com.intel.bigdata.management.agent.HadoopTaskAspect.aroundTaskRun(HadoopTaskAspect.java:95)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1531)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> ... 19 more
> Caused by: java.lang.RuntimeException: Error in configuring object
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 24 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> ... 27 more
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:142)
> ... 32 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException:
> Configuration and input path are inconsistent
> at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:419)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:110)
> ... 32 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Configuration and input path are inconsistent
> at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:413)
> ... 33 more
>
> I have try to figure out the problem, but I can't find out the reason. I ask anyone for help. Thanks very much.
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) [mailto:elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit
> [https://groups.google.com/d/msgid/elasticsearch/4bcaef49-7483-4777-a8a2-6af671a082af%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4bcaef49-7483-4777-a8a2-6af671a082af%40googlegroups.com)
> [https://groups.google.com/d/msgid/elasticsearch/4bcaef49-7483-4777-a8a2-6af671a082af%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/4bcaef49-7483-4777-a8a2-6af671a082af%40googlegroups.com?utm_medium=email&utm_source=footer).
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
Costin
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/532950E0.8040607%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/532950E0.8040607%40gmail.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:42am UTC](https://discuss.elastic.co/t/elasticsearch-hadoop-problem-run-hive-script-for-two-table-join-some-error-happened-two-tables-also-stored-in-elasticsearch/16449/3 "2017-07-06T01:42:18Z")