Spark streaming elasticsearch dependencies

Elastic Stack Elasticsearch
1

Im trying Spark & Elasticsearch integration in scala as described in Elasticsearch
Guide
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/master/spark.html

I have problems with dependencies when compiling :

[trace] Stack trace suppressed: run last :update for the full output.[error] (:update) sbt.ResolveException: unresolved dependency: cascading#ing-local;2.5.6: not found[error] unresolved dependency: clj-time#clj-time;0.4.1: not found[error] unresolved dependency: compojure#compojure;1.1.3: not found[error] unresolved dependency: hiccup#hiccup;0.3.6: not found[error] unresolved dependency: ring#ring-devel;0.3.11: not found[error] unresolved dependency: ring#ring-jetty-adapter;0.3.11: not found[error] unresolved dependency: com.twitter#carbonite;1.4.0: not found[error] unresolved dependency: cascading#cascading-hadoop;2.5.6: not found[error] Total time: 86 s, completed 19 nov. 2014 08:42:58

My build.sbt file look like this

name := "twitter-sparkstreaming-elasticsearch"

version := "0.0.1"

scalaVersion := "2.10.4"
// additional libraries
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "1.1.0",
"org.apache.spark" %% "spark-streaming" % "1.1.0",
"org.apache.spark" %% "spark-streaming-twitter" % "1.1.0",
"org.elasticsearch" % "elasticsearch-hadoop" % "2.1.0")

Help? thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/11105bb6-2148-4eef-bc43-d2b3fcc16f12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Cascading and its dependencies are not available in Maven central but
in their own repo (which es-hadoop cannot specify through its pom).

However since you are not using cascasding but rather spark, I suggest
you use the dedicated jar for that, namely: elasticsearch-spark_2.10

On 11/19/14, Mohamed Hedi ABIDI abidimh@gmail.com wrote:

Im trying Spark & Elasticsearch integration in scala as described in
Elasticsearch
Guide
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/master/spark.html

I have problems with dependencies when compiling :

[trace] Stack trace suppressed: run last :update for the full
output.[error] (:update) sbt.ResolveException: unresolved dependency:
cascading#ing-local;2.5.6: not found[error] unresolved dependency:
clj-time#clj-time;0.4.1: not found[error] unresolved dependency:
compojure#compojure;1.1.3: not found[error] unresolved dependency:
hiccup#hiccup;0.3.6: not found[error] unresolved dependency:
ring#ring-devel;0.3.11: not found[error] unresolved dependency:
ring#ring-jetty-adapter;0.3.11: not found[error] unresolved dependency:
com.twitter#carbonite;1.4.0: not found[error] unresolved dependency:
cascading#cascading-hadoop;2.5.6: not found[error] Total time: 86 s,
completed 19 nov. 2014 08:42:58

My build.sbt file look like this

name := "twitter-sparkstreaming-elasticsearch"

version := "0.0.1"

scalaVersion := "2.10.4"
// additional libraries
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "1.1.0",
"org.apache.spark" %% "spark-streaming" % "1.1.0",
"org.apache.spark" %% "spark-streaming-twitter" % "1.1.0",
"org.elasticsearch" % "elasticsearch-hadoop" % "2.1.0")

Help? thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/11105bb6-2148-4eef-bc43-d2b3fcc16f12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJogdmfe8bFtm_ZDR5QrUv5nOPOsbAeoqEL44xkPGL3kW6u85Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.