Unresolved dependency: org.pentaho

Elastic Stack Elasticsearch
1

I'm running emr-5.2.0 with Spark 2.0.2 and I'm getting an error when trying to use the new 5.1.1 release.
Any ideas why?

$ spark-shell --packages org.elasticsearch:elasticsearch-hadoop:5.1.1    
...
Exception in thread "main" java.lang.RuntimeException: [unresolved dependency: org.pentaho#pentaho-aggdesigner-algorithm;5.1.5-jhyde: not found, unresolved dependency: cascading#cascading-hadoop;2.6.3: not found, unresolved dependency: cascading#cascading-local;2.6.3: not found]
        	at org.apache.spark.deploy.SparkSubmitUtils$.resolveMavenCoordinates(SparkSubmit.scala:1076)
        	at org.apache.spark.deploy.SparkSubmit$.prepareSubmitEnvironment(SparkSubmit.scala:294)
        	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:158)
        	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:124)
        	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2

Looks like I have the wrong package. I was looking for the latest build on maven here https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-spark_2.10 and I didn't see the official 5.1.1 build. I ended up on the hadoop page and noticed the 5.1.1 version there and figured it was renamed to match the es-hadoop product name.

Anyways I found the package hosted on spark-packages here https://spark-packages.org/package/elastic/elasticsearch-hadoop

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.