How to determine "index.mapping.total_fields.limit" in an Index in ES 6.3

Hello,

PUT test1
{
"mappings": {
"_doc": {
"properties": {
"field1":{
"type":"text"
},
"field2":{
"type":"text"
},
"field3":{
"type":"text"
}
}
}
},
"settings": {
"analysis": {
"normalizer": {
"ExportRawAnalyzer": {
"filter": "lowercase",
"type": "custom",
"char_filter": {
"char_filter": "html_strip"
}
}
},
"analyzer": {
"ExportPrimaryAnalyzer": {
"filter": "lowercase",
"char_filter": "html_strip",
"type": "custom",
"tokenizer": "whitespace"
}
}
}
}
}

These are my index settings.

Now I ran this query:

PUT test1/_settings
{
"index.mapping.total_fields.limit": 3
}

I inserted this doc:

PUT test1/_doc/d1
{
"field1":"d1_f1value",
"field2":"d1_f2value",
"field3":"d1_f3value"
}

It is inserting properly.
But when I tried to insert

PUT test1/_doc/d2
{
"field1":"d2_f1value",
"field2":"d2_f2value",
"field3":"d2_f3value",
"field4":"d2_f4value"
}

I am getting and error of field limit exceeded.
Then I changed this to 16 as below:

PUT test1/_settings
{
"index.mapping.total_fields.limit": 16
}

Then the d2 doc is being inserted.

I want to know how this is being calculated. Why not

PUT test1/_settings
{
"index.mapping.total_fields.limit": 4
}
this is not working.

Hi,
Below is my analysis on this,

#Mapping for test1 when we have 1 field dynamic:
> {

  "test1": {
    "mappings": {
      "_doc": {
        "properties": {
          "field1": {
            "type": "text", 
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}

When Total: 1 field all dynamic: index.mapping.total_fields.limit = 13 (11(meta fields) + 2(for type dynamic))
When Total: 2 fields all dynamic:index.mapping.total_fields.limit = 15 (11(meta fields) +2(for type dynamic) +2(for type dynamic))

#Mapping for test2 when we have 1 static field but total_fields limit is '0' :

{
"test2": {
"aliases": {},
"mappings": {
"_doc": {
"properties": {
"field1": {
"type": "text"
}
}
}
},
"settings": {
"index": {
"mapping": {
"total_fields": {
"limit": "0"
}
},
"number_of_shards": "5",
"provided_name": "test5",
"creation_date": "1578395477727",
"number_of_replicas": "1",
"uuid": "LO_a8woySs216VkyQoVzmg",
"version": {
"created": "6030099"
}
}
}
}
}

If fields are defined in the index mappings(Not Dynamically) then limit setting is not considered at all.

#Mapping for test3 when we have 1 static field and 1 dynamic field:

{
"test3": {
"aliases": {},
"mappings": {
"_doc": {
"properties": {
"field1": {
"type": "text"
},
"field3": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"settings": {
"index": {
"mapping": {
"total_fields": {
"limit": "14"
}
},
"number_of_shards": "5",
"provided_name": "test4",
"creation_date": "1578393184484",
"number_of_replicas": "1",
"uuid": "NpL3Vn65RP-V126NK_Wzng",
"version": {
"created": "6030099"
}
}
}
}
}

When Total: 2 fields in which 1 dynamic :index.mapping.total_fields.limit = 14 (11(meta fields) +1(for type normal) + 2(for type dynamic))
When Total: 4 fields in which 1 dynamic :index.mapping.total_fields.limit = 16 (11(meta fields) +3(for type normal) +2(for type dynamic))
When Total: 5 fields in which 2 dynamic :index.mapping.total_fields.limit = 18 (11(meta fields) +3(for type normal) +2(for type dynamic)+2(for type dynamic))

So basically we have 11(meta fields) not sure which these fields exactly are but found this from
Doc: Clarify count for index.mapping.total_fields.limit · Issue #24096 · elastic/elasticsearch · GitHub
_routing
_ttl
_index
_type
_all
_parent
_field_names
_source
_id
_version
_timestamp
_uid

Here there are 12 fields, but currently I don't know which are there in these 11(meta fields) not sure about this.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.