Logstash keep creating field despite the dynamic_mapping being deactivated

Elastic Stack Logstash
1

I have defined my own template to be used by logstash where I have deactivate the dynamic mapping:

{
	"my_index": {
		"order": 0,
		"template": "my_index",
		"settings": {
			"index": {
				"mapper": {
					"dynamic": "false"
				},
				"analysis": {
					"analyzer": {
						"nlp_analyzer": {
							"filter": [
								"lowercase"
							],
							"type": "custom",
							"tokenizer": "nlp_tokenizer"
						}
					},
					"tokenizer": {
						"nlp_tokenizer": {
							"pattern": ""
							"(\w+)|(\s*[\s+])"
							"",
							"type": "pattern"
						}
					}
				},
				"number_of_shards": "1",
				"number_of_replicas": "0"
			}
		},
		"mappings": {
			"author": {
				"properties": {
					"author_name": {
						"type": "keyword"
					},
					"author_pseudo": {
						"type": "keyword"
					},
					"author_location": {
						"type": "text",
						"fields": {
							"standard": {
								"analyzer": "standard",
								"term_vector": "yes",
								"type": "text"
							},
							"nlp": {
								"analyzer": "nlp_analyzer",
								"term_vector": "yes",
								"type": "text"
							}
						}
					}
				}
			}
		}
	}
}

To test if elasticsearch won't generate new field I try to let a field in my events that is not present in my mapping, let's say that I have this event:

{
"type" => "author",
"author_pseudo" => "chloemdelorenzo",
"author_name" => "Chloe DeLorenzo",
"author_location" => "US",
}

Elasticsearch will generate a new fielding the mapping when indexing this event:

"type": {
     "type": "text",
      "fields": {
           "keyword": {
                "type": "keyword",
                "ignore_above": 256
           }
      }
 }

I know that Logstash is using my template because in my mapping I use a custom analyser and I can find it back into the mapping generated. But apparently it doesn't take into consideration that the dynamic field is disabled.

I want elastic search to ignore fields that are not present in my mapping but to index the field that have a defined mapping. How can I avoid logstash to create new field?

2

Have you checked the actual mappings of the index, thereby verifying that your template has been correctly applied?

3

Thanks! I have found my error:

I had to enforce the mapping at the document type level in the documentation they state this:

Regardless of the value of this setting, types can still be added explicitly when creating an index or with the PUT mapping API.

So my question now is what is the purpose of :

 "mapper": {
	   "dynamic": "false"
 }
4

Sorry, I don't understand the question.

5

Sorry, let me reformulate:

Apparently this settings doesn't deactivate the dynamic mapping:

"settings": {
			"index": {
				"mapper": {
					"dynamic": "false"
				},
                ....

But this does:

"mappings": {
    "author": {
        "dynamic": false,
        "properties": {
        ...

My question is why the first setting have no impact on the dynamic mapping? And what is it used for?

6

That setting is used to disable dynamic type creation.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.