Dynamic Templates updates as part of upgrade

[venkat_dinesh (DineshVenkat)]

Hi,

Context :
I am upgrading a set of indices with data from Elasticsearch 7.0 to Elasticsearch 7.10 (I do not have much control over the target version).

In some of existing indices, mappings use dynamic_templates. Example is as below :

{
  "event_collection" : {
    "mappings" : {
      "dynamic_templates" : [
        {
          "ips" : {
            "match" : "ip_addr_*",
            "match_mapping_type" : "string",
            "mapping" : {
              "type" : "ip"
            }
          }
        },
        {
          "dates" : {
            "match" : "date_*",
            "match_mapping_type" : "string",
            "mapping" : {
              "type" : "date"
            }
          }
        },
        {
          "strings" : {
            "match_mapping_type" : "string",
            "mapping" : {
              "norms" : false,
              "type" : "keyword"
            }
          }
        },
        {
          "longs" : {
            "match_mapping_type" : "long",
            "mapping" : {
              "norms" : false,
              "type" : "long"
            }
          }
        },
        {
          "doubles" : {
            "match_mapping_type" : "double",
            "mapping" : {
              "norms" : false,
              "type" : "double"
            }
          }
        }
      ],
      "date_detection" : false,
      "properties" : {
        "~eventid" : {
          "type" : "keyword"
        },
        "~evtlabel" : {
          "type" : "keyword"
        },
        "~event_report_time" : {
          "type" : "long"
        }
      }
    }
  }
}

Issue observed
I tried creating index with same mapping in Elasticsearch 7.10.2 and tried to index data. I started getting following exceptions for long and double fields and indexing was failing

MapperParsingException[unknown parameter [norms] on mapper [eventRating] of type [double]] Failure

Intermediary solution
Same exception seen on long fields too. Hence, I completely removed norms from doubles and and longs dynamic-templates and managed to successfully index the documents.
This implies that long. and double fields should not even have norms parameter. This would work for fresh deployments

Unfortunately, I have to figure-out what to do with existing indices from old-version and plan migration / upgrade procedure to allowcontinued data-ingestion of data into old-indices too.

Actual Problem and Clarifications
Since I already have indices created by old dynamic-template as per above example which contains norms param for double and long types

1. What would happen when old index from Elasticsearch 7.0 is backed-up and restored in Elasticsearch 7.10.2 ?

2. Can I update mappings using mapping-update API https://www.elastic.co/guide/en/elasticsearch/reference/7.10/indices-put-mapping.html to update dynamic_templates of old indices to eliminate norms from long and double types as part of my upgrade workflow before allowing indices to be available for data-ingestion ?

3. If update API is suitable, is there any example reference available as part of documentation - tried searching but could not find one ?

4. If update API is not a suitable approach, what are the alternate approaches to be taken as part of upgrade to avoid MapperParsingException ?

5. I assume that removal of norms as mentioned above would not require reindexing - as I understand norms is a score which is stored in index-segments and not part of the index-data itself and hence removal / disablement of norms from any fields would not require reindexing- please let me know if I am missing something

Thanks in advance

Dinesh