# Is there a guide from creating uber jar with spark core and elasticsearch-spark? **URL:** \ **Category:** Elasticsearch\ **Tags:** es-hadoop\ **Created:** [July 13, 2015, 9:44am UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427 "2015-07-13T09:44:03Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![HeartSaVioR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heartsavior/32/3670_2.png) [@HeartSaVioR](https://discuss.elastic.co/u/HeartSaVioR)\ **Post date:** [July 13, 2015, 9:44am UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/1 "2015-07-13T09:44:03Z")
Hi! I'm using Spark 1.4.0, and just start experimenting elasticsearch-hadoop. I don't want to let Spark adding some libraries, so I made a uber jar whenever I made a new drivers. I added "org.elasticsearch" %% "elasticsearch-spark" % "2.1.0" to build.sbt, and ran "sbt assembly", and met issue from deduplication. > java.lang.RuntimeException: deduplicate: different file contents found in the following: > /Users/heartsavior/.ivy2/cache/com.esotericsoftware.kryo/kryo/bundles/kryo-2.21.jar:com/esotericsoftware/minlog/Log$Logger.class > /Users/heartsavior/.ivy2/cache/com.esotericsoftware.minlog/minlog/jars/minlog-1.2.jar:com/esotericsoftware/minlog/Log$Logger.class I excluded spark-core from elasticsearch-spark with no luck. So, I'd like to know about best practice to exclude libraries so that I can maintain uber jar which contains elasticsearch-spark. Thanks in advance! ---
**Author:** ![angel\_ft](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/angel_ft/32/3654_2.png) [@angel\_ft](https://discuss.elastic.co/u/angel_ft)\ **Post date:** [July 13, 2015, 12:09pm UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/2 "2015-07-13T12:09:47Z")
Hi, I had the same problem and ended just adding elasticsearch .jar as an "unmanaged" dependency (just placing it on the /lib/ folder of my project). Hope that works for you too. ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [July 13, 2015, 4:16pm UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/3 "2015-07-13T16:16:37Z")
Btw, elasticsearch spark is available as a Spark [package](http://spark-packages.org/package/elastic/elasticsearch-hadoop) so when using Spark 1.2, you can simply specify it from the command line when submitting your job. ---
**Author:** ![HeartSaVioR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heartsavior/32/3670_2.png) [@HeartSaVioR](https://discuss.elastic.co/u/HeartSaVioR)\ **Post date:** [July 13, 2015, 8:59pm UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/4 "2015-07-13T20:59:22Z")
Does it mean that I can let elasticsearch-spark as "provided" and just link it when submitting? ---
**Author:** ![HeartSaVioR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heartsavior/32/3670_2.png) [@HeartSaVioR](https://discuss.elastic.co/u/HeartSaVioR)\ **Post date:** [July 14, 2015, 4:06am UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/5 "2015-07-14T04:06:00Z")
I changed dependency "elasticsearch-spark" as "provided", and specified elasticsearch-spark package when submitting, and finally it works! Thanks! ---
**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\ **Post date:** [July 14, 2015, 5:30am UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/6 "2015-07-14T05:30:03Z")
Glad to hear it. Can you please post sbt setup as post/git/gist ? it might be useful to others trying to do the same thing in the future. Thanks, ---
**Author:** ![HeartSaVioR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heartsavior/32/3670_2.png) [@HeartSaVioR](https://discuss.elastic.co/u/HeartSaVioR)\ **Post date:** [July 14, 2015, 8:33am UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/7 "2015-07-14T08:33:37Z")
Sure! Here it is. > ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:28pm UTC](https://discuss.elastic.co/t/is-there-a-guide-from-creating-uber-jar-with-spark-core-and-elasticsearch-spark/25427/8 "2017-07-06T13:28:03Z")