# Updating/upserting using Cascading/Scalding with a pre-defined \_id field

**URL:** <https://discuss.elastic.co/t/updating-upserting-using-cascading-scalding-with-a-pre-defined--id-field/23382>\
**Category:** Elasticsearch\
**Created:** [April 23, 2015, 8:34pm UTC](https://discuss.elastic.co/t/updating-upserting-using-cascading-scalding-with-a-pre-defined--id-field/23382 "2015-04-23T20:34:10Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andres\_Perez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andres_perez/32/757_2.png) [@Andres\_Perez](https://discuss.elastic.co/u/Andres_Perez)\
**Post date:** [April 23, 2015, 8:34pm UTC](https://discuss.elastic.co/t/updating-upserting-using-cascading-scalding-with-a-pre-defined--id-field/23382/1 "2015-04-23T20:34:10Z")

</div>

Hi. I've been creating Scalding Taps to read/write to elasticsearch (using  
cascading EsTaps underneath), and everything has been going fine when  
writing to documents using insert operations  
(es.write.operation=ES\_OPERATION\_INDEX). I am writing to a preexisting  
resource that I created through a REST call that defines the \_id field:

curl -XPUT [http://host](http://host):port/test\_index/?pretty -d '  
{  
"mappings": {  
"test\_document" : {  
"\_id" : {  
"path" : "some\_id\_field"  
}  
}  
}  
}  
'

This allows my EsTap to automatically use the "some\_id\_field" (from the  
cascading source) as the \_id when it gets written to elasticsearch.  
Unfortunately, when setting the es.write.operation property to  
ES\_OPERATION\_UPDATE or ES\_OPERATION\_UPSERT, it no longer automatically uses  
the some\_id\_field that I specified when I created the index. If I do not  
specify the field with the "es\_mapping\_id" property, I get either a

Operation [%s] requires an id but none was given/found

or

Operation [%s] requires an id but none (%s) was specified

error, depending on whether I'm trying to update or upsert.

My question is: is it possible to not have to specify a es\_mapping\_id  
property when updating/upserting (which would be less painful, since we  
aren't manually specifying fields when we load the data, anyway), or can I  
somehow retrieve the "path" value for \_id and pass it along to the  
cascading tap for any update/upsert operation?

Thanks,

Andy

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/a920e676-5de0-4cd5-b7b7-15b7d7f9b09f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/a920e676-5de0-4cd5-b7b7-15b7d7f9b09f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:18am UTC](https://discuss.elastic.co/t/updating-upserting-using-cascading-scalding-with-a-pre-defined--id-field/23382/2 "2017-07-06T00:18:00Z")

</div>