Mappings get messed up - Any way to enforce mappings

Elastic Stack Elasticsearch
1

I create my indices with a nice tidy mapping where most fields are keywords but, over time, the mapping mutates into a monster of text fields with a ".keyword" property. How can this happen and is there any way to stop it.

I start with:

{
    "logs-prod3": {
        "mappings": {
            "properties": {
                "category": {"type": "keyword"},
                "contact": {"type": "keyword"},
                ...

Then I re-index my data to that index and it mutates into:

{
"logs-prod3": {
    "mappings": {
        "properties": {
            "category": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                    }
                }
            },
            "contact": {
                "type": "text",
                "fields": {
                    "keyword": {
                        "type": "keyword",
                        "ignore_above": 256
                    }
                }
            },
            ...

Why is this happening?

How can I enforce a mapping - preferably reject any data not defined by the mapping!

2

Create an index template with the settings and mappings you want to apply to all matching indices.

3

According to the docs "templates are only applied at index creation time". Since I create the indices myself with the correct mapping and they get corrupted later I don't see how this can help. Indeed ES Docs claim mappings cannot be changed which makes me wonder how this happens at all.

4

You can disable dynamic mappings, but that will cause documents with new fields to fail.

5

The second index has a different name so does not get the settings from the first. This is where an index template is useful.

6

The mismatch of names is just a typo (copy-paste error) posting there. I created the indices with the keyword mapping, verified this and then re-indexed.

Thanks for the tip about dynamic mappings - sounds like exactly what I want!

7

btw. if I disable dynamic mappings, then add data with new unmapped fields... these are not indexed but stored in source. Can I update the mapping on the index and reindex the new _source data within the same index? I assume I have to make a new index but just checking...

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.