Dynamic Templates: path_match with regex matching producing unintentional results

[nkachami (Nicholas Kachami)]

Below is a mock index that contains a dynamic template utilizing path_match with regex to index specifically named objects into an Elasticsearch schema.

{
    "settings": {# settings
    },
    "mappings": {
        "dynamic": "strict",
        "dynamic_templates": [
            {
                "some_template": {
                    "match_pattern": "regex",
                    "path_match": "top_level_field.(sub_level_field1|sub_level_field2|sub_level_fieldx)..*",
                    "mapping": {
                        "type": "object",
                        "properties": {# identical properties of specifically named sub_level_fields
                        }
                    }
                }
            }
        ],
        "properties": {
            "top_level_field": {
                "type": "object",
                "dynamic": true
            },
            # more properties
        }
    }
}

With the template above, I should expect to be able to index:

{
  "top_level_field": {
    "sub_level_field1": {
      # properties
    }
  }
}

I should also expect not to be able to index

{
  "top_level_field": {
    "non_listed_sub_level_field": {
      # properties
    }
  }
}

But, both can be indexed. Does path_match not function the same way match does with regex or am I missing something?

Thanks

[demjened (Adam Demjen)]

Hello Nicholas, welcome to the forum!

path_match is not regex but takes a path syntax. Looking at your dynamic mapping definition I see an extra dot at the end of the expression that might be there by accident.

I tried setting up the index like this:

PUT dyntest
{
    "mappings": {
        "dynamic": "strict",
        "dynamic_templates": [
            {
                "some_template": {
                    "match_pattern": "regex",
                    "path_match": "top_level_field.(sub_level_field1|sub_level_field2|sub_level_fieldx).*", // just one dot in .*
                    "mapping": {
                        "type": "object",
                        "properties": {
                          "subfield": {
                            "type": "keyword"
                          }
                        }
                    }
                }
            }
        ],
        "properties": {
            "top_level_field": {
                "type": "object",
                "dynamic": true
            }
        }
    }
}

This way the some_template mapping applies to top_level_field.sub_level_field1.subfield, top_level_field.sub_level_field2.subfield and so on, and it will map it as keyword.

Since top_level_field is dynamic, it can take other sub-fields that do not match for the template, for those the default mapping will apply.

PUT dyntest/_doc/1
{
  "top_level_field": {
    "sub_level_field1": {
      "subfield": "foo"
    },
    "non_listed_sub_level_field": {
      "accepts_anything": true
    }
  }
}

GET dyntest/_mapping

...
"properties": {
  "top_level_field": {
    "dynamic": "true",
    "properties": {
      "non_listed_sub_level_field": {
        "properties": {
          "accepts_anything": {
            "type": "boolean" // Did not apply dynamic template
          }
        }
      },
      "sub_level_field1": {
        "properties": {
          "subfield": {
            "type": "keyword" // Applied dynamic template
          }
        }
      }
    }
  }
}

Hope I understood the problem correctly and this answers your question!

[nkachami (Nicholas Kachami)]

Thanks for the quick response Adam. This makes sense, I thought that template may limit the top level field to only accept a singular sub field or in this case the list of fields within the regex, but because its marked as dynamic, I see that is not the case. Thanks for the response!