ERROR: Multiple ES-Hadoop versions detected - Elastic Stack / Elasticsearch - Discuss the Elastic Stack
I met the same question as above.
If keep those versions, it occurs Multiple ES-Hadoop versions detected as follows:
Caused by: java.lang.RuntimeException: Multiple ES-Hadoop versions detected in the classpath; please use only one
if i remove one version, it occurs:
Previous writer likely failed to write hdfs://nameservice1/tmp/hive/hive/_tez_session_dir/bcde0ca7-3058-4670-a1e0-480de5264124-resources/elasticsearch-hadoop-7.0.0-rc1.jar. Failing because I am unlikely to write too.
How are you running hive, and how are you adding elasticsearch-hadoop to the classpath?
I runned hive sqls as belows on tez by using beeline client. Es version is 7.1.1, Hive version is cdp7.4.4, and I added "elasticsearch-hadoop-7.1.1.jar", "commons-httpclient-3.0.1.jar", "commons-httpclient-3.1.jar" in the aux path set in cloudera manager. By the way, it works on cdh6.3.2 by mr engine, but in cdp7.4.4, it supporst only tez engine, and missed commons-httpclient-3.0.1.jar.
DROP TABLE IF EXISTS test.history_deliver_es;
CREATE TABLE IF NOT EXISTS test.history_deliver_es( fund_account string,
rowkey string )
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.resource' = 'test/history_deliver_es',
'es.index.auto.create' = 'false',
"es.mapping.id" = "fund_account",
'es.mapping.names' = 'fund_account:fund_account, rowkey:rowkey',
'es.nodes' = '188.8.131.52:9200,184.108.40.206:9200,220.127.116.11:9200,18.104.22.168:9200,22.214.171.124:9200');
insert into table test.history_deliver_es values('02','20230613');
Solved. Restart yarn and hive after remove 7.0.0-rc1.jar in aux path and clean yarn cache dir. I think it related to cdp, maybe jar takes effect only after restarting. Thanks for your response.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.