Multiple ES-Hadoop versions detected in the classpath

ERROR: Multiple ES-Hadoop versions detected - Elastic Stack / Elasticsearch - Discuss the Elastic Stack

I met the same question as above.
If keep those versions, it occurs Multiple ES-Hadoop versions detected as follows:
Caused by: java.lang.RuntimeException: Multiple ES-Hadoop versions detected in the classpath; please use only one

if i remove one version, it occurs:
Previous writer likely failed to write hdfs://nameservice1/tmp/hive/hive/_tez_session_dir/bcde0ca7-3058-4670-a1e0-480de5264124-resources/elasticsearch-hadoop-7.0.0-rc1.jar. Failing because I am unlikely to write too.

How are you running hive, and how are you adding elasticsearch-hadoop to the classpath?

I runned hive sqls as belows on tez by using beeline client. Es version is 7.1.1, Hive version is cdp7.4.4, and I added "elasticsearch-hadoop-7.1.1.jar", "commons-httpclient-3.0.1.jar", "commons-httpclient-3.1.jar" in the aux path set in cloudera manager. By the way, it works on cdh6.3.2 by mr engine, but in cdp7.4.4, it supporst only tez engine, and missed commons-httpclient-3.0.1.jar.

DROP TABLE IF EXISTS test.history_deliver_es;

CREATE TABLE IF NOT EXISTS test.history_deliver_es( fund_account string,
                              rowkey string )
                              STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
                              TBLPROPERTIES('es.resource' = 'test/history_deliver_es',
'' = 'false',
"" = "fund_account",
'es.mapping.names' = 'fund_account:fund_account, rowkey:rowkey',
'es.nodes' = ',,,,');

insert into table test.history_deliver_es values('02','20230613');

Solved. Restart yarn and hive after remove 7.0.0-rc1.jar in aux path and clean yarn cache dir. I think it related to cdp, maybe jar takes effect only after restarting. Thanks for your response.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.