We currently load data to ES using a Spark script running on a Hortonworks cluster HDP 2.4 (Spark 1.6). We have a new dev ES cluster 5.2. I got this error when running the same script to load the data to ES 5.2 cluster:
org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Unsupported/Unknown Elasticsearch version 5.2.0
I checked the latest Hortonworks distro, and it is also using Spark 1.6.
Any way to use Hortonworks with ES 5.2?