ElasticSearch and JSON SparkSQL

Elastic Stack Elasticsearch
1

Hello,

Just curious, is this plugin using the JSON support of SparkSQL (I mean http://spark.apache.org/docs/latest/sql-programming-guide.html#json-datasets).

Currently I have always a trouble with ES schema, and as a workaround, I export my ES data as a big JSON file and load it using org.apache.spark.sql.json, here no problem about nested, arrays etc...

Thanks you,

2

No, it does not.
Sorry to hear about your issues, can you try ES-Hadoop 2.2-rc1 and report back if addresses your problem. If so, can you please post a sample of your JSON.

Thanks,