Multiple ES-Hadoop versions detected in the classpath

[Suna.Y]

ERROR: Multiple ES-Hadoop versions detected - Elastic Stack / Elasticsearch - Discuss the Elastic Stack

I met the same question as above.
If keep those versions, it occurs Multiple ES-Hadoop versions detected as follows:
Caused by: java.lang.RuntimeException: Multiple ES-Hadoop versions detected in the classpath; please use only one
jar:file:/home/app/yarn/nm/usercache/hive/filecache/2753/elasticsearch-hadoop-7.1.1.jar
jar:file:/home/app/yarn/nm/usercache/hive/filecache/2756/elasticsearch-hadoop-7.0.0-rc1.jar

if i remove one version, it occurs:
Previous writer likely failed to write hdfs://nameservice1/tmp/hive/hive/_tez_session_dir/bcde0ca7-3058-4670-a1e0-480de5264124-resources/elasticsearch-hadoop-7.0.0-rc1.jar. Failing because I am unlikely to write too.

[Keith_Massey]

How are you running hive, and how are you adding elasticsearch-hadoop to the classpath?

[Suna.Y]

I runned hive sqls as belows on tez by using beeline client. Es version is 7.1.1, Hive version is cdp7.4.4, and I added "elasticsearch-hadoop-7.1.1.jar", "commons-httpclient-3.0.1.jar", "commons-httpclient-3.1.jar" in the aux path set in cloudera manager. By the way, it works on cdh6.3.2 by mr engine, but in cdp7.4.4, it supporst only tez engine, and missed commons-httpclient-3.0.1.jar.

DROP TABLE IF EXISTS test.history_deliver_es;

CREATE TABLE IF NOT EXISTS test.history_deliver_es( fund_account string,
                              rowkey string )
                              STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
                              TBLPROPERTIES('es.resource' = 'test/history_deliver_es',
'es.index.auto.create' = 'false',
"es.mapping.id" = "fund_account",
'es.mapping.names' = 'fund_account:fund_account, rowkey:rowkey',
'es.nodes.wan.only'='true',
'es.batch.write.retry.count'='10',
'es.batch.write.refresh'='true',
'es.batch.write.retry.wait'='60s',
'es.http.timeout'='100m',
'es.batch.size.entries'='100',
'es.nodes' = '192.25.109.213:9200,192.25.109.214:9200,192.25.109.215:9200,192.25.109.216:9200,192.25.109.217:9200');

insert into table test.history_deliver_es values('02','20230613');

[Suna.Y]

Solved. Restart yarn and hive after remove 7.0.0-rc1.jar in aux path and clean yarn cache dir. I think it related to cdp, maybe jar takes effect only after restarting. Thanks for your response.