Elastic - spark : deprecated search_type=scan showing up in post url from spark 2.4 and returning 0 hits

Hi Team

Am trying to fetch data from elastic server into spark 2.4 environment using "org.elasticsearch.spark.sql" connector, but even though i got all the configs right, there is no data received on the spark engine

Using the below config to connect to Elastic from spark


On analyzing with fiddler found that the request created had a search_type=scan param in the query string, which i guess is no longer supported


On opening this on the browser didn't receive any data, but the same query without the search_type param worked as expected.

Am wondering when the above param is no longer supported, why is it being appended in the request url by the elasticsearch-spark connector

Can someone please help me figure out a proper solution

Spark - 2.4.0
scala - 2.11
elasticsearch-spark-20_2.11 (6.6.0)


@james.baiera Do you know?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.