How to disable dynamic mapping in 5.1.1?

Elastic Stack Elasticsearch
1

Running 5.1.1, followed the 5.1 docs and set a template and setting on the mapping definition and it still creates dynamic mappings.

Set template to disable dynamic mapping for all:

curl -XPUT 'http://localhost:9200/_template/template_all' -d '
{
  "template": "*",
  "order":0,
  "settings": {
    "index.mapper.dynamic": false 
  }
}'

Create mapping:

curl -XPUT 'http://localhost:9200/test_schema' -d '
{
  "mappings": {
    "test_type": {
      "properties": {
        "foo": {
          "type": "string"
        }
      }
    }
  }
}'

Set mapping to disable dynamic mapping (since setting template failed to disable):

curl -XPUT 'http://localhost:9200/test_schema/_settings' -d '
{
  "index.mapper.dynamic":false 
}'

Attempt to index data with missing field mapping definition, should fail, but doesn't:

curl -XPUT 'http://localhost:9200/test_schema/test_type/1' -d '
{
  "foo": "hello",
  "bar": "should cause error"
}'

Successful, but should have thrown error:

{"_index":"test_schema","_type":"test_type","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"created":true}

Get the mapping to see if it contains the dynamic mapping:

curl http://localhost:9200/test_schema/_mapping?pretty=true

Yep, sure does, but shouldn't:

dev@ip-172-31-10-217:~/workspace/generic-schema-scale-test$ curl http://localhost:9200/test_schema/_mapping?pretty=true
{
  "test_schema" : {
    "mappings" : {
      "test_type" : {
        "properties" : {
          "bar" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },
          "foo" : {
            "type" : "text"
          }
        }
      }
    }
  }
}
2

you want to look at https://www.elastic.co/guide/en/elasticsearch/reference/5.1/dynamic-field-mapping.html#dynamic-field-mapping to prevent new fields within a type from being added to mappings.

3

There was nothing on your linked page that was useful. As I said, I followed the 5.1 docs for both global (template) and per index settings and neither disabled dynamic mapping. Try the code I pasted above to reproduce and you will observe it isn't working.

4

I recreated the behavior, not being able to 'lock down' the addition of new fields to the schema, when adding new data or updating the schema. This seemed odd, but is explained as follows at the bottom of the docs:

Regardless of the value of this setting, types can still be added explicitly when creating an index or with the PUT mapping API.

I was able to slip in new schemas just by posting non-schema data ({"wam": "bam"}, for instance), and add to the test_schema index. This seemed adverse, but I was able to work around towards the behavior I think you were expecting, using the 'strict' mapping setting when creating custom types. In your example, test_type should be 'strict', causing an exception to be thrown when a field besides 'foo' is present.

To test, I followed your setup above, but added strict mapping to the test_type:

curl -XPUT 'http://localhost:9200/test_schema' -d '
{
  "mappings": {
    "test_type": {
      "dynamic": "strict",
      "properties": {
        "foo": {
          "type": "string"
        }
      }
    }
  }
}'

This prevented the next data from being indexed, instead throwing the exception:

Request:

curl -XPOST 'http://localhost:9200/test_schema/test_type/2' -d ' 
    {
      "wam": "bam"
    }'

Response:

{
  "error": {
    "root_cause": [
      {
        "type": "strict_dynamic_mapping_exception",
        "reason": "mapping set to strict, dynamic introduction of [wam] within [test_type] is not allowed"
      }
    ],
    "type": "strict_dynamic_mapping_exception",
    "reason": "mapping set to strict, dynamic introduction of [wam] within [test_type] is not allowed"
  },
  "status": 400
}
2 Likes
5

yeah so see @offshore comment below.

6

That's what I was expecting. This page in the ES 5.1 docs is still misleading/incorrect.

7

glad to hear we've resolved your query. feel free to open a PR if you feel like contributing to our docs. thanks

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.