I am trying to use Jest client in spark , its working good only if I use in single machine, if I use in cluster mode, its not working due to Client is not serialized. Any one using java client which is serialized ?
@sukumar Have you considered using the native Spark integration that we have in ES-Hadoop? We support regular Spark RDD's, Spark Streaming DStreams and SparkSQL DataFrames/Datasets, and we're compatible against all Elasticsearch versions 1.x and up.
@james.baiera I tried with the native spark integration and I am not seeing the fast response , spark and ES are in two different clusters, its taking too long to get response. Now I am using jersey client to make rest call in the spark transformation and works fine and i can make multiple request in parallel.
Hi Sukumar,
I am starting using JEST for my application. Could you please share your experience to me?
thanks,
Zhifeng