Update Document in Elasticsearch using spark 1.6


(Gunjit Bansal) #1

Hi,

How to update any document in elasticsearch using spark 1.6


(Mark Walkom) #2

What have you tried?


(Gunjit Bansal) #3

Hi,

I tried few things, one of them is

val res = d.drop("col1").withColumn("col2",lit("a")).withColumn("_id",lit("AWENjnCoOLO5K8P91uWK"))
EsSpark.saveJsonToEs(res.toJSON,"master_file/master",Map("es.mapping.id","_id"))


(Gunjit Bansal) #4

Hi,

I am able to update the document using case class. But I don't want to add elasticsearch id in my document. How can i achieve this ?


(James Baiera) #5

You could try setting es.mapping.exclude to the name of the id field you are using (docs). WHen used with the es.mapping.id option, the connector will still use the id in the row as the document's id, but not to serialize it as part of the actual document.


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.