Spark-shell sc.esRDD fails with error: unrecognized parameter: [_local]

Elastic Stack Elasticsearch
1

I am using elasticsearch-spark-13_2.10-5.0.0-beta1.jar for spark 1.6 and elasticsearch version 5.0.2

I am trying the very basic code:

val lpiRDD = sc.esRDD("sdp/lpi")
lpiRDD.count

And getting following error

Caused by: org.elasticsearch.hadoop.rest.EsHadoopInvalidRequest: request [/sdp/lpi/_search] contains unrecognized parameter: [_local]
{"query":{"match_all":{}}}
at org.elasticsearch.hadoop.rest.RestClient.checkResponse(RestClient.java:488)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:446)
at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:436)
at org.elasticsearch.hadoop.rest.RestRepository.scroll(RestRepository.java:363)
at org.elasticsearch.hadoop.rest.ScrollQuery.hasNext(ScrollQuery.java:92)

Please help.

Thanks a lot
Apratim

2

It was jar compatibility issue. Unfortunately it's really hard to find correct version of jar for elastic-spark.
I found jar for my versions spark 1.6 scala 2.10 and es 5.0.2 at https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-spark-13_2.10/5.0.2

After that it worked just fine.

Thanks

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.