# Unable to compile spark 2.0 project with es-hadoop

**URL:** <https://discuss.elastic.co/t/unable-to-compile-spark-2-0-project-with-es-hadoop/98569>\
**Category:** Elasticsearch\
**Tags:** es-hadoop\
**Created:** [August 28, 2017, 1:08pm UTC](https://discuss.elastic.co/t/unable-to-compile-spark-2-0-project-with-es-hadoop/98569 "2017-08-28T13:08:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Noorul\_Islam\_Kamal\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noorul_islam_kamal_m/32/21574_2.png) [@Noorul\_Islam\_Kamal\_M](https://discuss.elastic.co/u/Noorul_Islam_Kamal_M)\
**Post date:** [August 28, 2017, 1:08pm UTC](https://discuss.elastic.co/t/unable-to-compile-spark-2-0-project-with-es-hadoop/98569/1 "2017-08-28T13:08:43Z")

</div>

I have the following dependency added in sbt

```
"org.elasticsearch" %% "elasticsearch-spark-20" % "5.5.2"

```

I get the following error

[warn] Run 'evicted' to see detailed eviction warnings  
[info] Compiling 1 protobuf files to /opt/aruba/central/apps/mon-ae-notifications/alert\_detector/target/src\_managed/main  
[info] Compiling schema /opt/aruba/central/apps/mon-ae-notifications/alert\_detector/src/main/protobuf/notifications.proto  
protoc-jar: protoc version: 320, detected platform: linux/amd64  
protoc-jar: executing: [/tmp/protocjar2523904227258434471/bin/protoc.exe, --plugin=protoc-gen-scala=/tmp/protocbridge1934192420034201130, --scala\_out=grpc:/opt/aruba/central/apps/mon-ae-notifications/alert\_detector/target/src\_managed/main, -I/opt/aruba/central/apps/mon-ae-notifications/alert\_detector/src/main/protobuf, -I/opt/aruba/central/apps/mon-ae-notifications/alert\_detector/target/protobuf\_external, /opt/aruba/central/apps/mon-ae-notifications/alert\_detector/src/main/protobuf/notifications.proto]  
[info] Compiling protobuf  
[info] Protoc target directory: /opt/aruba/central/apps/mon-ae-notifications/alert\_detector/target/src\_managed/main  
[info] Compiling 15 Scala sources to /opt/aruba/central/apps/mon-ae-notifications/alert\_detector/target/classes...  
[error] missing or invalid dependency detected while loading class file 'StreamingContext.class'.  
[error] Could not access term hadoop in package org.apache,  
[error] because it (or its dependencies) are missing. Check your build definition for  
[error] missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)  
[error] A full rebuild may help if 'StreamingContext.class' was compiled against an incompatible version of org.apache.  
[error] missing or invalid dependency detected while loading class file 'StreamingContext.class'.  
[error] Could not access type Configuration in value org.apache.conf,  
[error] because it (or its dependencies) are missing. Check your build definition for  
[error] missing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)  
[error] A full rebuild may help if 'StreamingContext.class' was compiled against an incompatible version of org.apache.conf.  
[error] two errors found  
[error] (AlertDetector/compile:compileIncremental) Compilation failed  
[error] Total time: 67 s, completed 16 Aug, 2017 9:42:15 PM

---

<div class="post-metadata">

**Author:** ![james.baiera](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james.baiera/32/10209_2.png) [@james.baiera](https://discuss.elastic.co/u/james.baiera)\
**Post date:** [September 12, 2017, 3:09pm UTC](https://discuss.elastic.co/t/unable-to-compile-spark-2-0-project-with-es-hadoop/98569/2 "2017-09-12T15:09:25Z")

</div>

> [@Noorul\_Islam\_Kamal\_M](#):
>
> [error] Could not access term hadoop in package org.apache,
> 
> [error] because it (or its dependencies) are missing. Check your build definition for
> 
> [error] missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.)

This may be the issue. We exclude Hadoop from our dependencies. You should try adding an explicit dependency on Hadoop.

---

<div class="post-metadata">

**Author:** ![Noorul\_Islam\_Kamal\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/noorul_islam_kamal_m/32/21574_2.png) [@Noorul\_Islam\_Kamal\_M](https://discuss.elastic.co/u/Noorul_Islam_Kamal_M)\
**Post date:** [September 13, 2017, 8:19am UTC](https://discuss.elastic.co/t/unable-to-compile-spark-2-0-project-with-es-hadoop/98569/3 "2017-09-13T08:19:37Z")

</div>

You are right, I fixed this by including hadoop dependencies but I forgot to close this thread.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 11, 2017, 8:20am UTC](https://discuss.elastic.co/t/unable-to-compile-spark-2-0-project-with-es-hadoop/98569/4 "2017-10-11T08:20:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.