Enable .keyword fields for specific fields only

Elastic Stack Elasticsearch
1

Hi,

Elasticsearch Version: 7.15.0
Logstash Version: 7.15.0

We have a ton of fields for our index and we get new fields frequently so, we cannot disable dynamic mapping. Dynamic mapping creates .keyword fields for a majority of the fields and we don't want that for ALL fields. By default we want to disable creating .keyword fields for all fields and enable that feature for specific fields. Can someone please tell us how we can achieve that?

FYI: we tried the following config on the index template (hoping that just request.* fields will get .keyword field) but it didn't work, we noticed that ALL .keyword fields are removed including request.* fields:

  "mappings" : {
      "dynamic_templates": [
      {
        "strings": {
          "match_mapping_type": "string",
          "path_unmatch": "request.*",
          "mapping": {
            "type": "text"
          }
        }
      },
      {
        "request_url": {
          "match_mapping_type": "string",
          "path_match": "request.*",
          "mapping": {
            "type": "keyword"
          }
        }
      }
    ]
  },
  • Thank you
2

I think there is some confusion here, you are mixing up the name of the field with the mapping of the field.

If you do not create a mapping for a field, Elasticsearch will create a mapping for this field when it receives the first document, for string fields it will map the field first as a text field and then it will map the same field again as a keyword field, this is done using multi-fields.

What happens is that, for example, if you have an unmapped field named ExampleField, and let Elasticsearch create the mapping, this is what you will have in the mapping:

        "ExampleField": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }

The parte after fields is the multi-field, what it is doing is mapping the same field on a different way with the suffix keyword.

So you have ExampleField as a text field and Example.keyword as a keyword field.

Since you are explicitly telling Elasticsearch to map fields starting with request.* as keyword, you will not have any field with the suffix .keyword, but all the request.* field will be mapped as keyword field.

1 Like
3

Thank you very much for a speedy reply.
I guess my next question is:

  • How do I enable both text and keyword fields for request.* fields? right now we are only getting keyword fields, we want a text field like request.* and keyword field request.*.keyword as well.

  • Thank you

4

this worked for me:

      "dynamic_templates": [
      {
        "strings": {
          "match_mapping_type": "string",
          "path_unmatch": "request.*",
          "match": "*",
          "mapping": {
            "type": "text"
          }
        }
      },
      {
        "request_url": {
          "match_mapping_type": "string",
          "path_match": "request.*",
          "mapping": {
            "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
          }
        }
      }
    ]
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.