Regexp in dynamic templates


#1

Hello,

I would like to create dynamic mapping so that all strings matching IP address pattern are mapped to IP type and all other strings are mapped to keyword/text.
I've tried the following index template, but it does not seem to catch IP addresses.

{
  "aliases": {},
  "mappings": {
    "doc": {
      "_all": {
        "enabled": false
      },
      "dynamic_templates": [
        {
          "strings": {
            "mapping": {
              "type": "keyword",
              "ignore_above": 256,
              "fields": {
                "raw": {
                  "type": "text"
                }
              }
            },
            "unmatch": "((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)",
            "match_pattern": "regex",
            "match_mapping_type": "string"
          }
        },
        {
          "ip_address": {
            "match_mapping_type": "string",
            "match_pattern": "regex",
            "match": "((25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)",
            "mapping": {
              "type": "ip"
            }
          }
        }
      ],
      "_source": {
        "enabled": true
      }
    }
  },
  "settings": {
    "index": {
      "query": {
        "default_field": "uri"
      },
      "number_of_replicas": "1",
      "auto_expand_replicas": "false",
      "translog": {
        "durability": "async"
      },
      "number_of_shards": "8",
      "refresh_interval": "60s"
    }
  },
  "index_patterns": [
    "haproxy-*"
  ],
  "order": 100
}

I did not find any documentations/examples, so I would appreciate any help on this.
Also, is there a way to debug dynamic_template rules?
Thank you very much.


#2

Anyone?


(Junaid) #3

Match_pattern / Match works over field names rather than field values. See the link for details.

It would be better to define templates similar to below mentioned snippet. So any field names matching ip_* pattern will automatically be applied with IP mapping type.

{  "aliases": {},  "mappings": {    "doc": {      "_all": {        "enabled": false      },      "dynamic_templates": [    {          "ip_address": {            "match_mapping_type": "string", "match": "ip_*",            "mapping": {              "type": "ip"            }          }        }      ],      "_source": {        "enabled": true      }    }  },  "settings": {    "index": {      "query": {        "default_field": "uri"      },      "number_of_replicas": "1",      "auto_expand_replicas": "false",      "translog": {        "durability": "async"      },      "number_of_shards": "3",      "refresh_interval": "60s"    }  },  "index_patterns": ["haproxy*"],  "order": 100}

If you are naming fields as IP addresses, it is not a good idea because you'll soon face mapping explosions, as IPv4 alone can have roughly 2^32 combinations.

Hope this answers your question.


#4

Thank you very much for taking the time and explaining this to me.
You are absolutely correct that I wanted to match field values (rather than names), which is apparently not supported.