# How to fix guava version conflicts with Hadoop YARN classpath? **URL:** \ **Category:** Elasticsearch\ **Created:** [March 24, 2016, 6:44pm UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384 "2016-03-24T18:44:45Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![royce-pandora](https://avatars.discourse-cdn.com/v4/letter/r/977dab/32.png) [@royce-pandora](https://discuss.elastic.co/u/royce-pandora)\ **Post date:** [March 24, 2016, 6:44pm UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/1 "2016-03-24T18:44:45Z")
Hi, I've spent several hours trying to fix a problem with Elastic Search v2.2 Java lib and Cloudera Hadoop Cluster v5.5.1. The problem: Guava-18.0.jar is not being recognized by elastic search v2.2 when used by a Pig (v0.11.0) UDF. As a result, the stack trace below occurs. I've read and implemented the To Shade or not To Shade article solution, however, I continue to get the exception error below. How do I make use of guava-18.0.jar for elastic search v2.2 while other hadoop projects use older guava versions in the class path? 2016-03-24 17:52:17,450 ERROR [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.directExecutor()Ljava/util/concurrent/Executor; at org.elasticsearch.threadpool.ThreadPool.(ThreadPool.java:190) at org.elasticsearch.client.transport.TransportClient$Builder.build(TransportClient.java:133) at com.nextbigsound.find.FindConfig.getElasticSearchClient(FindConfig.java:176) at com.nextbigsound.hadoop.analytics.find.FindTracksStoreFunction.prepareToWrite(FindTracksStoreFunction.java:252) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.(PigOutputFormat.java:125) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.getRecordWriter(PigOutputFormat.java:86) at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.(ReduceTask.java:540) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) ---
**Author:** ![Amandeep\_Singh](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@Amandeep\_Singh](https://discuss.elastic.co/u/Amandeep_Singh)\ **Post date:** [April 15, 2016, 11:44pm UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/2 "2016-04-15T23:44:19Z")
Having exactly the same issue. ---
**Author:** ![Amandeep\_Singh](https://avatars.discourse-cdn.com/v4/letter/a/49beb7/32.png) [@Amandeep\_Singh](https://discuss.elastic.co/u/Amandeep_Singh)\ **Post date:** [April 16, 2016, 12:08am UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/3 "2016-04-16T00:08:13Z")
Just solved it, apparently the hadoop version I was running the bulk loader had an earlier version of guava jar in the path. I was able to solve it by setting `mapreduce.job.user.classpath.first` to true Point being, make sure there is no other guava jar in the path than you supply ---
**Author:** ![royce-pandora](https://avatars.discourse-cdn.com/v4/letter/r/977dab/32.png) [@royce-pandora](https://discuss.elastic.co/u/royce-pandora)\ **Post date:** [April 19, 2016, 4:57pm UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/4 "2016-04-19T16:57:41Z")
The way I solved this issue is by using the [es-hadoop](https://www.elastic.co/guide/en/elasticsearch/hadoop/current/index.html) connector by elastic search. It avoids the guava version issue entirely and, in my case, replaces a custom elastic search implementation. I'll write a blog post and share how I solved the issue using Pig and es-hadoop. ---
**Author:** ![Sunil\_G.C](https://avatars.discourse-cdn.com/v4/letter/s/779978/32.png) [@Sunil\_G.C](https://discuss.elastic.co/u/Sunil_G.C)\ **Post date:** [June 20, 2016, 5:50am UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/5 "2016-06-20T05:50:43Z")
How and where to set? Please tell in detail. ---
**Author:** ![SaurabhM](https://avatars.discourse-cdn.com/v4/letter/s/8c91f0/32.png) [@SaurabhM](https://discuss.elastic.co/u/SaurabhM)\ **Post date:** [September 1, 2016, 9:09pm UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/6 "2016-09-01T21:09:16Z")
We also have the same issue any further details. ---
**Author:** ![Sunil\_G.C](https://avatars.discourse-cdn.com/v4/letter/s/779978/32.png) [@Sunil\_G.C](https://discuss.elastic.co/u/Sunil_G.C)\ **Post date:** [September 2, 2016, 1:41am UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/7 "2016-09-02T01:41:36Z")
Issue can be fix by relocating guava jar by maven and use guava 18.0 jar mention on pom.xml file. ---
**Author:** ![nmalinip](https://avatars.discourse-cdn.com/v4/letter/n/ce73a5/32.png) [@nmalinip](https://discuss.elastic.co/u/nmalinip)\ **Post date:** [September 2, 2016, 7:15pm UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/8 "2016-09-02T19:15:53Z")
We are facing same issue....Can you provide more details on how to set . Here is my build.sbt libraryDependencies ++= Seq( "org.apache.spark" %% "spark-core" % sparkVersion % "provided", "org.apache.spark" %% "spark-streaming" % sparkVersion % "provided", "org.apache.spark" %% "spark-streaming-kafka" % sparkVersion, "org.apache.spark" %% "spark-sql" % sparkVersion % "provided", "org.elasticsearch" % "elasticsearch-hadoop" % "5.0.0-alpha4", "org.elasticsearch" % "elasticsearch" % "2.3.4", "org.elasticsearch.plugin" % "shield" % "2.3.4" from "[https://maven.elasticsearch.org/releases/org/elasticsearch/plugin/shield/2.3.4/shield-2.3.4.jar](https://maven.elasticsearch.org/releases/org/elasticsearch/plugin/shield/2.3.4/shield-2.3.4.jar)", "joda-time" % "joda-time" % "2.7", "com.databricks" %% "spark-xml" % "0.3.3", "com.sun.jersey" % "jersey-servlet" % "1.19", "com.google.guava" % "guava" % "18.0", "com.amazonaws" % "aws-java-sdk" % "1.11.26", "com.typesafe" % "config" % "1.3.0", "com.databricks" %% "spark-csv" % "1.4.0", "org.apache.spark" %% "spark-mllib" % sparkVersion ) resolvers ++= Seq( "Akka Repository" at "[http://repo.akka.io/releases/](http://repo.akka.io/releases/)", "scala-tools" at "[https://oss.sonatype.org/content/groups/scala-tools](https://oss.sonatype.org/content/groups/scala-tools)" //"elasticsearch-releases" at "[https://maven.elasticsearch.org/releases](https://maven.elasticsearch.org/releases)" ) ---
**Author:** ![Sunil\_G.C](https://avatars.discourse-cdn.com/v4/letter/s/779978/32.png) [@Sunil\_G.C](https://discuss.elastic.co/u/Sunil_G.C)\ **Post date:** [September 3, 2016, 5:43am UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/9 "2016-09-03T05:43:36Z")
Please refer to this link [here](https://www.elastic.co/blog/to-shade-or-not-to-shade) to relocate the jar. Thanks, ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:22pm UTC](https://discuss.elastic.co/t/how-to-fix-guava-version-conflicts-with-hadoop-yarn-classpath/45384/10 "2017-07-05T22:22:42Z")