# Indexing and "\_id" question **URL:** \ **Category:** Elasticsearch\ **Created:** [September 30, 2010, 6:07pm UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392 "2010-09-30T18:07:15Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![dmcclure1](https://avatars.discourse-cdn.com/v4/letter/d/f19dbf/32.png) [@dmcclure1](https://discuss.elastic.co/u/dmcclure1)\ **Post date:** [September 30, 2010, 6:07pm UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392/1 "2010-09-30T18:07:15Z")
If you have an "\_id" field in your source json and try to index it without passing in an id of your own, as in prepareIndex("indexName", "type"), you get an error complaining that the "\_id" generated by elastic does not match "\_id" in source. Same goes if I pass in my own id to prepareSearch() to use for the index id and source contains an "\_id" that does not match up. I know this is expected behavior but was curious if there was a way around it, possibly be able to map "\_id" to another field in source, ignore existing "\_id" in source, or use "\_id" already in source if it exists and an index id is not passed to prepareIndex()? Background: We are using mongodb, converting mongodb objects into json and storing them into elastic. Mongo also uses \_id and under the covers this is a org.bson.types.ObjectId, which basically creates an \_id field like: ``` "_id":{"time": ``` 1285864090000,"new":true,"machine":-561909349,"inc":1450029131} when converted to json. If this field is used at top-level object for source then elastic will error out for non-matching ids. Can't have elastic use this id as is for index id, would need to convert to GUID, which is not a problem, just curious if there was a way around it. Thanks! ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 30, 2010, 6:13pm UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392/2 "2010-09-30T18:13:11Z")
Hey, No, there isn't currently a way around this. It really simplifies things when the \_id, \_type, and \_source can't "change" their names. I actually started (way back) with allowing that, but it got things too complicated to be worth the effort. It also makes maintaing things much simpler. I think you will have to handle it on your end..., sorry. -shay.banon On Thu, Sep 30, 2010 at 8:07 PM, dmcclure [duane.mcclure@gmail.com](mailto:duane.mcclure@gmail.com) wrote: > If you have an "\_id" field in your source json and try to index it > without passing in an id of your own, as in prepareIndex("indexName", > "type"), you get an error complaining that the "\_id" generated by > elastic does not match "\_id" in source. Same goes if I pass in my own > id to prepareSearch() to use for the index id and source contains an > "\_id" that does not match up. > > I know this is expected behavior but was curious if there was a way > around it, possibly be able to map "\_id" to another field in source, > ignore existing "\_id" in source, or use "\_id" already in source if it > exists and an index id is not passed to prepareIndex()? > > Background: > We are using mongodb, converting mongodb objects into json and storing > them into elastic. Mongo also uses \_id and under the covers this is a > org.bson.types.ObjectId, which basically creates an \_id field like: > > ``` > "_id":{"time": > > ``` > > 1285864090000,"new":true,"machine":-561909349,"inc":1450029131} > > when converted to json. If this field is used at top-level object for > source then elastic will error out for non-matching ids. Can't have > elastic use this id as is for index id, would need to convert to GUID, > which is not a problem, just curious if there was a way around it. > > Thanks! ---
**Author:** ![dmcclure1](https://avatars.discourse-cdn.com/v4/letter/d/f19dbf/32.png) [@dmcclure1](https://discuss.elastic.co/u/dmcclure1)\ **Post date:** [September 30, 2010, 6:17pm UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392/3 "2010-09-30T18:17:42Z")
No worries at all, figured that was case but thought I'd ask just to be sure. Thanks again! On Sep 30, 2:13 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Hey, > > No, there isn't currently a way around this. It really simplifies things > when the \_id, \_type, and \_source can't "change" their names. I actually > started (way back) with allowing that, but it got things too complicated to > be worth the effort. It also makes maintaing things much simpler. I think > you will have to handle it on your end..., sorry. > > -shay.banon > > On Thu, Sep 30, 2010 at 8:07 PM, dmcclure [duane.mccl...@gmail.com](mailto:duane.mccl...@gmail.com) wrote: > > > If you have an "\_id" field in your source json and try to index it > > without passing in an id of your own, as in prepareIndex("indexName", > > "type"), you get an error complaining that the "\_id" generated by > > elastic does not match "\_id" in source. Same goes if I pass in my own > > id to prepareSearch() to use for the index id and source contains an > > "\_id" that does not match up. > > > I know this is expected behavior but was curious if there was a way > > around it, possibly be able to map "\_id" to another field in source, > > ignore existing "\_id" in source, or use "\_id" already in source if it > > exists and an index id is not passed to prepareIndex()? > > > Background: > > We are using mongodb, converting mongodb objects into json and storing > > them into elastic. Mongo also uses \_id and under the covers this is a > > org.bson.types.ObjectId, which basically creates an \_id field like: > > > ``` > > "_id":{"time": > > > > ``` > > > > 1285864090000,"new":true,"machine":-561909349,"inc":1450029131} > > > when converted to json. If this field is used at top-level object for > > source then elastic will error out for non-matching ids. Can't have > > elastic use this id as is for index id, would need to convert to GUID, > > which is not a problem, just curious if there was a way around it. > > > Thanks! ---
**Author:** ![Mahendra\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mahendra_m/32/3128_2.png) [@Mahendra\_M](https://discuss.elastic.co/u/Mahendra_M)\ **Post date:** [October 1, 2010, 4:37am UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392/4 "2010-10-01T04:37:29Z")
Hi Shay, One quick question. How is this being handled in the CouchDB river plan ? couchdb also has an '\_id' field by default. Regards, Mahendra On Thu, Sep 30, 2010 at 11:43 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Hey, > > No, there isn't currently a way around this. It really simplifies things > when the \_id, \_type, and \_source can't "change" their names. I actually > started (way back) with allowing that, but it got things too complicated to > be worth the effort. It also makes maintaing things much simpler. I think > you will have to handle it on your end..., sorry. > > -shay.banon > > On Thu, Sep 30, 2010 at 8:07 PM, dmcclure [duane.mcclure@gmail.com](mailto:duane.mcclure@gmail.com) wrote: > > > If you have an "\_id" field in your source json and try to index it > > without passing in an id of your own, as in prepareIndex("indexName", > > "type"), you get an error complaining that the "\_id" generated by > > elastic does not match "\_id" in source. Same goes if I pass in my own > > id to prepareSearch() to use for the index id and source contains an > > "\_id" that does not match up. > > > > I know this is expected behavior but was curious if there was a way > > around it, possibly be able to map "\_id" to another field in source, > > ignore existing "\_id" in source, or use "\_id" already in source if it > > exists and an index id is not passed to prepareIndex()? > > > > Background: > > We are using mongodb, converting mongodb objects into json and storing > > them into elastic. Mongo also uses \_id and under the covers this is a > > org.bson.types.ObjectId, which basically creates an \_id field like: > > > > ``` > > "_id":{"time": > > > > ``` > > > > 1285864090000,"new":true,"machine":-561909349,"inc":1450029131} > > > > when converted to json. If this field is used at top-level object for > > source then elastic will error out for non-matching ids. Can't have > > elastic use this id as is for index id, would need to convert to GUID, > > which is not a problem, just curious if there was a way around it. > > > > Thanks! -- Mahendra [http://twitter.com/mahendra](http://twitter.com/mahendra) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [October 1, 2010, 10:10am UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392/5 "2010-10-01T10:10:48Z")
It maps nicely no? The same one is used for elasticsearch and couchdb. On Fri, Oct 1, 2010 at 6:37 AM, Mahendra M [mahendra.m@gmail.com](mailto:mahendra.m@gmail.com) wrote: > Hi Shay, > > One quick question. How is this being handled in the CouchDB river plan ? > couchdb also has an '\_id' field by default. > > Regards, > Mahendra > > On Thu, Sep 30, 2010 at 11:43 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > > wrote: > > > Hey, > > > > No, there isn't currently a way around this. It really simplifies things > > when the \_id, \_type, and \_source can't "change" their names. I actually > > started (way back) with allowing that, but it got things too complicated to > > be worth the effort. It also makes maintaing things much simpler. I think > > you will have to handle it on your end..., sorry. > > > > -shay.banon > > > > On Thu, Sep 30, 2010 at 8:07 PM, dmcclure [duane.mcclure@gmail.com](mailto:duane.mcclure@gmail.com)wrote: > > > > > If you have an "\_id" field in your source json and try to index it > > > without passing in an id of your own, as in prepareIndex("indexName", > > > "type"), you get an error complaining that the "\_id" generated by > > > elastic does not match "\_id" in source. Same goes if I pass in my own > > > id to prepareSearch() to use for the index id and source contains an > > > "\_id" that does not match up. > > > > > > I know this is expected behavior but was curious if there was a way > > > around it, possibly be able to map "\_id" to another field in source, > > > ignore existing "\_id" in source, or use "\_id" already in source if it > > > exists and an index id is not passed to prepareIndex()? > > > > > > Background: > > > We are using mongodb, converting mongodb objects into json and storing > > > them into elastic. Mongo also uses \_id and under the covers this is a > > > org.bson.types.ObjectId, which basically creates an \_id field like: > > > > > > ``` > > > "_id":{"time": > > > > > > ``` > > > > > > 1285864090000,"new":true,"machine":-561909349,"inc":1450029131} > > > > > > when converted to json. If this field is used at top-level object for > > > source then elastic will error out for non-matching ids. Can't have > > > elastic use this id as is for index id, would need to convert to GUID, > > > which is not a problem, just curious if there was a way around it. > > > > > > Thanks! > > -- > Mahendra > > [http://twitter.com/mahendra](http://twitter.com/mahendra) ---
**Author:** ![Mahendra\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mahendra_m/32/3128_2.png) [@Mahendra\_M](https://discuss.elastic.co/u/Mahendra_M)\ **Post date:** [October 1, 2010, 10:27am UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392/6 "2010-10-01T10:27:31Z")
Oh yes! /me is duh! 🙂 On Fri, Oct 1, 2010 at 3:40 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > It maps nicely no? The same one is used for elasticsearch and couchdb. > > On Fri, Oct 1, 2010 at 6:37 AM, Mahendra M [mahendra.m@gmail.com](mailto:mahendra.m@gmail.com) wrote: > > > Hi Shay, > > > > One quick question. How is this being handled in the CouchDB river plan ? > > couchdb also has an '\_id' field by default. > > > > Regards, > > Mahendra > > > > On Thu, Sep 30, 2010 at 11:43 PM, Shay Banon \< > > [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)\> wrote: > > > > > Hey, > > > > > > No, there isn't currently a way around this. It really simplifies > > > things when the \_id, \_type, and \_source can't "change" their names. I > > > actually started (way back) with allowing that, but it got things too > > > complicated to be worth the effort. It also makes maintaing things much > > > simpler. I think you will have to handle it on your end..., sorry. > > > > > > -shay.banon > > > > > > On Thu, Sep 30, 2010 at 8:07 PM, dmcclure [duane.mcclure@gmail.com](mailto:duane.mcclure@gmail.com)wrote: > > > > > > > If you have an "\_id" field in your source json and try to index it > > > > without passing in an id of your own, as in prepareIndex("indexName", > > > > "type"), you get an error complaining that the "\_id" generated by > > > > elastic does not match "\_id" in source. Same goes if I pass in my own > > > > id to prepareSearch() to use for the index id and source contains an > > > > "\_id" that does not match up. > > > > > > > > I know this is expected behavior but was curious if there was a way > > > > around it, possibly be able to map "\_id" to another field in source, > > > > ignore existing "\_id" in source, or use "\_id" already in source if it > > > > exists and an index id is not passed to prepareIndex()? > > > > > > > > Background: > > > > We are using mongodb, converting mongodb objects into json and storing > > > > them into elastic. Mongo also uses \_id and under the covers this is a > > > > org.bson.types.ObjectId, which basically creates an \_id field like: > > > > > > > > ``` > > > > "_id":{"time": > > > > > > > > ``` > > > > > > > > 1285864090000,"new":true,"machine":-561909349,"inc":1450029131} > > > > > > > > when converted to json. If this field is used at top-level object for > > > > source then elastic will error out for non-matching ids. Can't have > > > > elastic use this id as is for index id, would need to convert to GUID, > > > > which is not a problem, just curious if there was a way around it. > > > > > > > > Thanks! > > > > -- > > Mahendra > > > > [http://twitter.com/mahendra](http://twitter.com/mahendra) -- Mahendra [http://twitter.com/mahendra](http://twitter.com/mahendra) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:18am UTC](https://discuss.elastic.co/t/indexing-and--id-question/3392/7 "2017-07-06T04:18:37Z")