Not able to detect ES version

[sheryl]

Exception in thread "main" org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'
i am dumping dataframe to elastic search but i am not able to do it .

df.write.format("org.elasticsearch.spark.sql")
.option("es.nodes.wan.only","true")
.option("es.port","9200")
.option("es.net.ssl","true")
.option("es.nodes","localhost")
.mode("append")
.option("es.nodes.client.only", "false")
.save("C:\elasticsearch-6.3.1")

[james.baiera]

I see two items in your configuration that stand out to me. First: I see that you have es.net.ssl set to true. SSL/TLS encryption requires a licensed version of Elasticsearch, unless you are using a third party plugin that provides that encryption. Please make sure that you have SSL set up correctly, since if it is not configured right it will keep the connector from being able to reach Elasticsearch. You should be able to view more information about the connection failure in the logs. If you have SSL configured, usually that is paired with access controls. Make sure to add your credentials to the job, as described in the docs.

While the log message suggests configuring the es.nodes.wan.only setting, it is a good idea to be familiar with what that setting is doing in ES-Hadoop. That will disable node discovery and node filtering which generally is not advised unless you are running ES in the cloud.

After you sort out your connection issues, the second thing you will probably run into is that your elasticsearch index is not a valid index name:

@sheryl

.save("C:\elasticsearch-6.3.1")

That string is going to be used as the index in Elasticsearch to write data to. You will most likely want to change that setting to be something more appropriate for your index name.