how i can do bulk inserts to Elasticsearch in spark using scala?
I believe when you read a JSON or any datasource into Spark, the write operation into Elasticsearch may be like bulk insert (to some extent). As every executor with-in the spark cluster is going to run the saveToEs action in parallel.
If you would like to use non-spark APIs to perform a bulk insert, I have used akka-http for this. Not sure if this may be an overkill for you, as akka-streams may have a bit of a learning curve.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.