# Help with nested datatype mapping

**URL:** <https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818>\
**Category:** Elasticsearch\
**Created:** [June 17, 2017, 8:25pm UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818 "2017-06-17T20:25:37Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 17, 2017, 8:25pm UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/1 "2017-06-17T20:25:37Z")

</div>

Hi, I'm having problems with the schema for a nested datatype.

So far, I have the following:

```
"mappings": {
	"note": {
		"_all": {
			"enabled": false
		},
		"properties": {
			"user_id": {
				"type": "long"
			},
			"creation": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"deleted": {
				"type": "integer"
			},
			"favourite": {
				"type": "integer"
			},
			"modification": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"note": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"title": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"links_to_asset": {
				"type": "nested",
				"properties": {
					"note_link_id": {
						"type": "long"
					},
					"user_id": {
						"type": "long"
					},
					"creation": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"modification": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"to_asset": {
						"type": "long"
					},
					"from_asset": {
						"type": "long"
					},
					"comment": {
						"type": "text",
						"analyzer": "english",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "asset_en_analyzer",
								"fields": {
									"std": {
										"type": "text",
										"analyzer": "standard"
									}
								}
							}
						}
					}
				}
			}
		}
	}
}

```

Here, the `links_to_asset` appears to be of the proper format (based on the few examples I've been able to find).

However, when I populate Elasticsearch with data, the `"type": "nested"` attribute is removed.

Does anyone know how I could fix this?

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 19, 2017, 2:42am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/2 "2017-06-19T02:42:03Z")

</div>

I am not following what you mean by `"type": "nested" attribute is removed`. Is it being removed from the mapping? If not, what makes you think it is being removed? Please, give me a sample of what you are observing that makes you think it is being removed.

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 19, 2017, 7:56am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/3 "2017-06-19T07:56:15Z")

</div>

Hi @thiago, that's correct.

Once data is added and I check the mappings, that attribute has been removed.

```
{
    "asset_en_v1": {
        "mappings": {}
    },
    "asset": {
        "mappings": {
            "note": {
                "properties": {
                    "creation": {
                        "type": "date"
                    },
                    "deleted": {
                        "type": "long"
                    },
                    "favourite": {
                        "type": "long"
                    },
                    "links_to_asset": {
                        "properties": {
                            "comment": {
                                "type": "text",
                                "fields": {
                                    "keyword": {
                                        "type": "keyword",
                                        "ignore_above": 256
                                    }
                                }
                            },
                            "creation": {
                                "type": "date"
                            },
                            "from_asset": {
                                "type": "long"
                            },
                            "modification": {
                                "type": "date"
                            },
                            "note_link_id": {
                                "type": "long"
                            },
                            "to_asset": {
                                "type": "long"
                            },
                            "user_id": {
                                "type": "long"
                            }
                        }
                    },
                    "modification": {
                        "type": "date"
                    },
                    "note": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "title": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    },
                    "user_id": {
                        "type": "long"
                    }
                }
            }
        }
    }
}
```

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 19, 2017, 9:04am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/4 "2017-06-19T09:04:08Z")

</div>

Is this for a new index? Elasticsearch won't do that unless you are indexing in a new index. Are you using templates?

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 19, 2017, 9:28am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/5 "2017-06-19T09:28:21Z")

</div>

Hi @thiago, this is a new index.

No, I'm not using templates; I'm running the mapping via Postman.

I've had similar problems with date attributes, when the data was the wrong format, so I'm wondering if this is the same, thought I don't see what's wrong with the data.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 19, 2017, 9:32am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/6 "2017-06-19T09:32:54Z")

</div>

So you need to create the mapping in the new index before indexing. You should also index in the same type defined in the index mapping, otherwise Elasticsearch will try to autodetect the mapping and it won't autodetect nested types.

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 19, 2017, 10:19am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/7 "2017-06-19T10:19:32Z")

</div>

Hi @thiago, that's what I'm doing.

Elasticsearch is performing some some autodetection in spite of the fact I'm specifying the index with exact values before indexing data.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 19, 2017, 10:40am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/8 "2017-06-19T10:40:43Z")

</div>

What is the name of the index that you are trying to create and also could you give a sample document that you are trying to index after you create the index?

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 19, 2017, 11:02am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/9 "2017-06-19T11:02:19Z")

</div>

Hi @thiago, the index is: `asset_en_v1` and I'm using the alias: `asset` to point to it.

It's not feasible to give you a sample document as the process of indexing is automated (I'm indexing existing data from within a database).

After testing, the index and the mappings are correct throughout the application, with the exception of the nested datatype.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 19, 2017, 11:25am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/10 "2017-06-19T11:25:06Z")

</div>

Are you sure the alias is properly setup? After you index it, verify that you can query the document in `asset` but also in `asset_en_v1`.

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 19, 2017, 11:48am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/11 "2017-06-19T11:48:25Z")

</div>

Hi @thiago, here's the workflow I go through:

DEL localhost:9200/\_all

PUT localhost:9200/asset\_en\_v1

```
{
	"settings": {
		"analysis": {
			"char_filter": {
				"&_to_and": {
					"type": "mapping",
					"mappings": ["&=> and "]
				}
			},
			"filter": {
				"asset_en_stopwords": {
					"type": "stop",
					"stopwords": ["_english_"]
				},
				"asset_en_stemmer": {
					"type": "stemmer",
					"name": "english"
				},
				"asset_en_shingle": {
					"type": "shingle",
					"max_shingle_size": 5,
					"min_shingle_size": 2,
					"output_unigrams": false,
					"output_unigrams_if_no_shingles": true
				}
			},
			"analyzer": {
				"asset_en_analyzer": {
					"type": "custom",
					"char_filter": ["html_strip", "&_to_and"],
					"tokenizer": "standard",
					"filter": ["asset_en_stopwords", "asset_en_stemmer", "lowercase", "asset_en_shingle", "asciifolding"]
				}
			}
		}
	}
},
"mappings": {
	"note": {
		"_all": {
			"enabled": false
		},
		"properties": {
			"user_id": {
				"type": "long"
			},
			"creation": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"deleted": {
				"type": "integer"
			},
			"favourite": {
				"type": "integer"
			},
			"modification": {
				"type": "date",
				"format": "date_hour_minute_second"
			},
			"note": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"title": {
				"type": "text",
				"analyzer": "english",
				"fields": {
					"std": {
						"type": "text",
						"analyzer": "asset_en_analyzer",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "standard"
							}
						}
					}
				}
			},
			"links_to_asset": {
				"type": "nested",
				"properties": {
					"note_link_id": {
						"type": "long"
					},
					"user_id": {
						"type": "long"
					},
					"creation": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"modification": {
						"type": "date",
						"format": "date_hour_minute_second"
					},
					"to_asset": {
						"type": "integer"
					},
					"from_asset": {
						"type": "integer"
					},
					"comment": {
						"type": "long",
						"analyzer": "english",
						"fields": {
							"std": {
								"type": "text",
								"analyzer": "asset_en_analyzer",
								"fields": {
									"std": {
										"type": "text",
										"analyzer": "standard"
									}
								}
							}
						}
					}
				}
			}
		}
	}

```

... other asset types (removed due to character length constraints).

```
}

```

PUT localhost:9200/asset\_en\_v1/\_alias/asset

Then I index the data from within the application.

As I said, the mapping appears to be correct, with the exception of the nested datatype.

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 19, 2017, 12:27pm UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/12 "2017-06-19T12:27:44Z")

</div>

It seems fine. Please, attach the full output of `GET asset_en_v1/_mapping`. Do not remove anything, if it's too big, then paste the content in [pastebin.com](http://pastebin.com) and link it here.

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 19, 2017, 2:39pm UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/13 "2017-06-19T14:39:37Z")

</div>

Hi @thiago:

> {  
> "asset\_en\_v1": {  
> "mappings": {}  
> }  
> }

I assume the alias isn't working?

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 19, 2017, 3:11pm UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/14 "2017-06-19T15:11:27Z")

</div>

So if you are indexing in `asset` and `asset_en_v1` mapping is like this it is because the alias command didn't work.

What is the output of `PUT /asset_en_v1/_alias/asset` command?

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 19, 2017, 3:34pm UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/15 "2017-06-19T15:34:39Z")

</div>

Hi @thiago:

```
{
    "acknowledged": true
} 

```

If it had been anything else, I would have mentioned it.

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 21, 2017, 9:25am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/16 "2017-06-21T09:25:15Z")

</div>

Hi @thiago, any further thoughts on this?

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [June 21, 2017, 2:27pm UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/17 "2017-06-21T14:27:50Z")

</div>

There should be some command failing in between. I don't understand how you get an empty mapping with `GET asset_en_v1/_mapping` while your command for creating the index `asset_en_v1` is defining mappings properly. Unless the create command is failing or the index is being deleted in between, I don't see how you can get an empty mapping.

---

<div class="post-metadata">

**Author:** ![waynesmallman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/waynesmallman/32/18596_2.png) [@waynesmallman](https://discuss.elastic.co/u/waynesmallman)\
**Post date:** [June 22, 2017, 9:59am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/18 "2017-06-22T09:59:29Z")

</div>

Hi @thiago, we found that the problem was in the formatting of the settings and mappings as one lump of code; Postman allowed the JSON to be executed in spite that it wasn't valid.

Thank you for your time — much appreciated!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 20, 2017, 9:59am UTC](https://discuss.elastic.co/t/help-with-nested-datatype-mapping/89818/19 "2017-07-20T09:59:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.