Hey,
Our complete three node cluster keeps on becoming completely unresponsive. The index concerned is up and running on one node, but isn't able to replicate to the other two nodes due to a mapping issue, apparently.
This is the issue
Mapper for [swiped_price] conflicts with existing mapping in other types[object mapping [swiped_price] can't be changed from non-nested to nested]
I understand what it's saying, but how do I resolve the issue?
Please help!
Ro.
Yoltsro,
The Found team pinged me (and others) on this - Having a look now, may ask for more info in a few.
Ro,
I hope you can bear with me. Can you help me understand your workflow better?
How/when did you create this index? What version of ES did you create it in? Did you upgrade your ES version afterward?
Do you have multiple types defined for the index?
Can provide the output of a call to GET /_mappings?
We created this index around 1 week ago, I think. We created it in ES 2.0. We haven't upgraded our version of ES.
We have two types defined in the index: products and interactions:
Hi @yoitsro
You have two fields called "swiped_price", one is of type "object" and the other is of type "nested". This kind of mapping conflict is no longer allowed in 2.0. Fields with the same name should have the same mapping, so you need to change one of those fields. I'm guessing that "swiped_price" can contain arrays of objects, in which case "nested" is more than likely the one you want to use.
See https://www.elastic.co/guide/en/elasticsearch/reference/current/nested.html for more on nested fields.
You should also delete the "_id" mapping - it is no longer configurable (although you're using the default "string" type and it will be accepted).
Hi Clinton,
Thanks for that. That swiped_price field on the products type was an accidental insert of a document which automatically created that field. However, I can now no longer delete it. The only other option is to reindex our data, but with our current situation as it is is (seemingly random failures), I can't do this. I've already reindexed the majority of the products collection into the mallzee_v4 index, but am very nervous about reindexing the interactions documents.
@yoitsro Are you saying that you created the swiped_price mapping in the interaction type with type nested, then indexed a document into "products" and it introduced the new "swiped_price" object mapping dynamically?
Do you have any dynamic mapping rules?
Ro,
What version of ES was the _v3 index created under?
I'd like to get to the bottom of how _v3 got to its current state, because, as Clinton explains, it contains mappings which violate constraints present in ESv2.0+.
I've just tried creating this index on 1.7.0, but it throws the same exception when I try to upgrade to 2.0 (as it should)
That is what happened. I don't have any dynamic mapping rules as of yet, but will need to add them once we can get to the bottom of this.
The _v3 index was created in ESv2.0. Like I say, the swiped_price was added dynamically.
swiped_price was added dynamically to _v3 products, correct? It's defined as nested on _v3 interactions, and nested mappings don't happen dynamically (unless maybe you have an index template. Do you?)
OK something is very weird:
products.swiped_price.* have custom mappings, not the ones that would be added by dynamic mapping without any custom rules.swiped_price field was added dynamically by indexing a document into the index, but it also exists in the _v4 index (which has the same custom mappings). Do you have a product document that contains swiped_price that you reindexed?I've tried following the steps you describe, but I get the expected exception... Any more details that you can provide to help me recreate this?
Correct - swiped_price was added dynamically to _v3 products. We don't have an index template.
Because of the error, I deleted the document containing the swiped_price field and tried to update the mapping on products to be nested. ES replied with acknowledged.
I'm going to need more details of what happened when, so I can try to recreate this.
Yeah, just mentioned this in the other reply - I did try to add the field after I saw the errors.
In the _v4 index, I've update the mapping so that swiped_price is no longer nested to prevent against potential future issues. I will also turn off dynamic mapping on _v4 shortly.
In order to reproduce, try the following:
Create the _v3 index with the attached mapping, BUT remove the swiped_price field from the products mapping.
Insert a document into products matching that format:
{
"name": "Aces Plain Overshirt",
"pid": "acesplainovershirt",
"type": "shirts",
"sku": "1041551812",
"description": "Workwear with a twist! This season, we’ve updated our versatile artisan overshirt. Unlined and crafted from a great herringbone cotton for an authentic utility feel, it also features two useful hand pockets. \n\nUtility Workwear Overshirt\nConstructed from a durable herringbone cotton",
"variant_link_id": "9b7823f16fe81e91031f711a104fb688",
"images": [
"http://cdn.shopify.com/s/files/1/0302/5117/products/RE0353P_Navy_1.jpg?v=1440145870",
"http://cdn.shopify.com/s/files/1/0302/5117/products/RE0353P_Navy_3.jpg?v=1440145870",
"http://cdn.shopify.com/s/files/1/0302/5117/products/RE0353P_Navy_2.jpg?v=1440145870"
],
"brand": {
"_id": "5579a4ddaf95c4cd2f93c12d",
"name": "Realm and Empire"
},
"parent_id": "55dcff6a8856831600188f61",
"primary_category": "outerwear",
"uniform_sort_time": "2015-08-24T15:35:20.260Z",
"buy_url": "http://www.realmandempire.com/collections/outerwear/products/aces-plain-overshirt-navy-re0353p",
"enabled": true,
"sizes": [
{
"_id": "3982334532",
"size": "x small",
"quantity": 15,
"price": {
"currency": "GBP",
"current": 120,
"on_sale": false
}
},
{
"_id": "3982334596",
"size": "small",
"quantity": 15,
"price": {
"currency": "GBP",
"current": 120,
"on_sale": false
}
},
{
"_id": "3982334660",
"size": "medium",
"quantity": 15,
"price": {
"currency": "GBP",
"current": 120,
"on_sale": false
}
},
{
"_id": "3982334724",
"size": "large",
"quantity": 15,
"price": {
"currency": "GBP",
"current": 120,
"on_sale": false
}
},
{
"_id": "3982334788",
"size": "x large",
"quantity": 15,
"price": {
"currency": "GBP",
"current": 120,
"on_sale": false
}
},
{
"_id": "3982334852",
"size": "2x large",
"quantity": 15,
"price": {
"currency": "GBP",
"current": 120,
"on_sale": false
}
}
],
"created_at": "2015-08-25T23:47:23.456Z",
"variants": [],
"interactions": [],
"suggest": [
"shirts",
"Realm and Empire",
"Realm & Empire",
"Realm and Empire shirts",
"navy shirts",
"navy Realm and Empire shirts"
],
"product_url": "http://www.realmandempire.com/collections/outerwear/products/aces-plain-overshirt-navy-re0353p",
"region": "GBR",
"colour": {
"hex": "#51769f",
"name": "navy",
"original": "navy"
},
"gender": "male",
"merchant": {
"_id": "5579a24f8b22b92caa95fa2c",
"name": "Realm & Empire"
},
"updated_at": "2015-11-21T10:23:43.188Z",
"stock_id": "navy"
}
Then update the document so it just contains the sizes (this happened accidentally for a few products).
Then update the document again so it contains the swiped_price field:
"swiped_price": {
"currency": "GBP",
"current": 5.99
}
This is how I got to this state.
Got it, that's a nasty bug! I'll open an issue to get that fixed, thanks for the recreation.
(Btw, I think you've done the wrong thing in v4... you WANT swiped_price to be nested, not object)
Question is how to get you out of this state...
How many shards do you have? What are you seeing in the logs?
The cluster's just gone down again 
Is there any way to get the cluster responsive again easily?
We have 6 shards and the logs keep complaining about the mapping conflict.
I'll change the mapping and reindex, assuming the cluster won't fall over again. Is there another way we can do this which isn't likely to take down the cluster?
Honestly, I don't know how to get you out of this state quickly without deleting the index.
How many docs are we talking about? Few enough to be able to reindex before the cluster is likely to fall over again? If not, either delete the index and lose the data or consider starting up a new cluster and reindexing the data bit by bit as you can...
Can you paste what you're seeing in the logs? Not just the exception we already know about. I'm looking for why the cluster goes down.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.