If you have an "_id" field in your source json and try to index it
without passing in an id of your own, as in prepareIndex("indexName",
"type"), you get an error complaining that the "_id" generated by
elastic does not match "_id" in source. Same goes if I pass in my own
id to prepareSearch() to use for the index id and source contains an
"_id" that does not match up.
I know this is expected behavior but was curious if there was a way
around it, possibly be able to map "_id" to another field in source,
ignore existing "_id" in source, or use "_id" already in source if it
exists and an index id is not passed to prepareIndex()?
Background:
We are using mongodb, converting mongodb objects into json and storing
them into elastic. Mongo also uses _id and under the covers this is a
org.bson.types.ObjectId, which basically creates an _id field like:
when converted to json. If this field is used at top-level object for
source then elastic will error out for non-matching ids. Can't have
elastic use this id as is for index id, would need to convert to GUID,
which is not a problem, just curious if there was a way around it.
No, there isn't currently a way around this. It really simplifies things
when the _id, _type, and _source can't "change" their names. I actually
started (way back) with allowing that, but it got things too complicated to
be worth the effort. It also makes maintaing things much simpler. I think
you will have to handle it on your end..., sorry.
If you have an "_id" field in your source json and try to index it
without passing in an id of your own, as in prepareIndex("indexName",
"type"), you get an error complaining that the "_id" generated by
elastic does not match "_id" in source. Same goes if I pass in my own
id to prepareSearch() to use for the index id and source contains an
"_id" that does not match up.
I know this is expected behavior but was curious if there was a way
around it, possibly be able to map "_id" to another field in source,
ignore existing "_id" in source, or use "_id" already in source if it
exists and an index id is not passed to prepareIndex()?
Background:
We are using mongodb, converting mongodb objects into json and storing
them into elastic. Mongo also uses _id and under the covers this is a
org.bson.types.ObjectId, which basically creates an _id field like:
when converted to json. If this field is used at top-level object for
source then elastic will error out for non-matching ids. Can't have
elastic use this id as is for index id, would need to convert to GUID,
which is not a problem, just curious if there was a way around it.
No, there isn't currently a way around this. It really simplifies things
when the _id, _type, and _source can't "change" their names. I actually
started (way back) with allowing that, but it got things too complicated to
be worth the effort. It also makes maintaing things much simpler. I think
you will have to handle it on your end..., sorry.
If you have an "_id" field in your source json and try to index it
without passing in an id of your own, as in prepareIndex("indexName",
"type"), you get an error complaining that the "_id" generated by
elastic does not match "_id" in source. Same goes if I pass in my own
id to prepareSearch() to use for the index id and source contains an
"_id" that does not match up.
I know this is expected behavior but was curious if there was a way
around it, possibly be able to map "_id" to another field in source,
ignore existing "_id" in source, or use "_id" already in source if it
exists and an index id is not passed to prepareIndex()?
Background:
We are using mongodb, converting mongodb objects into json and storing
them into elastic. Mongo also uses _id and under the covers this is a
org.bson.types.ObjectId, which basically creates an _id field like:
when converted to json. If this field is used at top-level object for
source then elastic will error out for non-matching ids. Can't have
elastic use this id as is for index id, would need to convert to GUID,
which is not a problem, just curious if there was a way around it.
No, there isn't currently a way around this. It really simplifies things
when the _id, _type, and _source can't "change" their names. I actually
started (way back) with allowing that, but it got things too complicated to
be worth the effort. It also makes maintaing things much simpler. I think
you will have to handle it on your end..., sorry.
If you have an "_id" field in your source json and try to index it
without passing in an id of your own, as in prepareIndex("indexName",
"type"), you get an error complaining that the "_id" generated by
elastic does not match "_id" in source. Same goes if I pass in my own
id to prepareSearch() to use for the index id and source contains an
"_id" that does not match up.
I know this is expected behavior but was curious if there was a way
around it, possibly be able to map "_id" to another field in source,
ignore existing "_id" in source, or use "_id" already in source if it
exists and an index id is not passed to prepareIndex()?
Background:
We are using mongodb, converting mongodb objects into json and storing
them into elastic. Mongo also uses _id and under the covers this is a
org.bson.types.ObjectId, which basically creates an _id field like:
when converted to json. If this field is used at top-level object for
source then elastic will error out for non-matching ids. Can't have
elastic use this id as is for index id, would need to convert to GUID,
which is not a problem, just curious if there was a way around it.
No, there isn't currently a way around this. It really simplifies things
when the _id, _type, and _source can't "change" their names. I actually
started (way back) with allowing that, but it got things too complicated to
be worth the effort. It also makes maintaing things much simpler. I think
you will have to handle it on your end..., sorry.
If you have an "_id" field in your source json and try to index it
without passing in an id of your own, as in prepareIndex("indexName",
"type"), you get an error complaining that the "_id" generated by
elastic does not match "_id" in source. Same goes if I pass in my own
id to prepareSearch() to use for the index id and source contains an
"_id" that does not match up.
I know this is expected behavior but was curious if there was a way
around it, possibly be able to map "_id" to another field in source,
ignore existing "_id" in source, or use "_id" already in source if it
exists and an index id is not passed to prepareIndex()?
Background:
We are using mongodb, converting mongodb objects into json and storing
them into elastic. Mongo also uses _id and under the covers this is a
org.bson.types.ObjectId, which basically creates an _id field like:
when converted to json. If this field is used at top-level object for
source then elastic will error out for non-matching ids. Can't have
elastic use this id as is for index id, would need to convert to GUID,
which is not a problem, just curious if there was a way around it.
No, there isn't currently a way around this. It really simplifies
things when the _id, _type, and _source can't "change" their names. I
actually started (way back) with allowing that, but it got things too
complicated to be worth the effort. It also makes maintaing things much
simpler. I think you will have to handle it on your end..., sorry.
If you have an "_id" field in your source json and try to index it
without passing in an id of your own, as in prepareIndex("indexName",
"type"), you get an error complaining that the "_id" generated by
elastic does not match "_id" in source. Same goes if I pass in my own
id to prepareSearch() to use for the index id and source contains an
"_id" that does not match up.
I know this is expected behavior but was curious if there was a way
around it, possibly be able to map "_id" to another field in source,
ignore existing "_id" in source, or use "_id" already in source if it
exists and an index id is not passed to prepareIndex()?
Background:
We are using mongodb, converting mongodb objects into json and storing
them into elastic. Mongo also uses _id and under the covers this is a
org.bson.types.ObjectId, which basically creates an _id field like:
when converted to json. If this field is used at top-level object for
source then elastic will error out for non-matching ids. Can't have
elastic use this id as is for index id, would need to convert to GUID,
which is not a problem, just curious if there was a way around it.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
