Change mapping of a list of indices (rollover)

Hi,
I'm using Elastic 7.11
I have a series of indices (not DataStreams) which rollover through an ILM policy.
There's a legacy index template which refer to them.

I would like to change the mapping of the future indices.

I tried to :

  1. Change the mapping attached to the template
     PUT _template/<name_of_my_legacy_index_template>
     {
        "settings" : { ...},
        "index_patterns" :  ["my_indices*"],
        "mappings" : { ...}
     }
  1. forced manually a rollover

However the new mappings didn't appear in the new index.

Question :
Is there a way to apply the new mapping to the future indices, without doing it every time (manually) the index is rollovered ?

Thank you !

This might indicate a problem with your template, can you post the entire thing?

Could you also share the mapping for the index that has been created after the rollover?

Hi !

A. First question, is it the right way to change the future index mapping

  1. Change the mapping at the template level ;
  2. the make a rollover.

It seems that the mapping in the new index, is not shown as long as no document is indexed. And even when the index is populated, only the fields given appear in the mapping.
This confuses me, because when the mapping is given through the Kibana interface (Index management > Legacy Templates) and fields given one by one ; the mapping (through the console) appears entirely and correctly.

B. Second question :

is there a way to "force" the mapping without using "dynamic: false or strict".
The mapping seems to be used very losely :
even if I describe a field as integer, I can still populate the index with integer AND text for the same filed described as integer in the mapping.

Here's my very simple example :

==================

PUT   _template/samia_test_mapping_template 
{
    "order" : 100,
    "version" : 1,
    "index_patterns" : ["samia_test_mapping_*"],
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "ilm_standard_1W",
          "rollover_alias" : "samia_test_mapping_alias"
        },
        "number_of_shards" : "1",
        "number_of_replicas" : "1"
      }
    },
    "mappings" : {
      "dynamic_templates": [],
      "properties" : {
        "phoenix_version": {
          "type": "constant_keyword",
          "value": "1.6"
        },
        "firstname" : {
          "type" : "text"
        }
      }
    },
    "aliases" : { }
}

Then I create the 1st index of the series :

PUT %3Csamia_test_mapping-%7Bnow%2Fd%7D-000001%3E 
{
  "aliases": {
    "samia_test_mapping_alias": {"is_write_index": true}
  }
}

Response :


{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "samia_test_mapping-2021.05.03-000001"
}


POST samia_test_mapping_alias/_doc
{
  "firstname": "samia"
}


Then I lookup at my first index mapping :


GET samia_test_mapping_alias/_mapping

Response : 

{
  "samia_test_mapping-2021.05.03-000001" : {
    "mappings" : {
      "properties" : {
        "firstname" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

As you can see, the mapping of the "firstname" has slightly changed.
And even if I set order to 1; I still have the same behaviour.

Then I change my template mapping and make a rollover :


PUT _template/samia_test_mapping_template 
{
  "index_patterns": ["samia_test_mapping_*"],
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1,
    "index": {
      "lifecycle": {
        "name": "ilm_standard_1W",
        "rollover_alias": "samia_test_mapping"
      }
    }
  }, 
  "mappings": {
    "properties": {
    "phoenix_version": {
      "type": "constant_keyword",
      "value" : "1.6"
    },
     "firstname": {"type": "text"},
     "name": {"type": "text"},
     "lastname": {"type" : "text"},
     "zipcode": {"type" : "integer"}
    } 
  }
}

POST samia_test_mapping_alias/_rollover

Response : 

{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "old_index" : "samia_test_mapping-2021.05.03-000001",
  "new_index" : "samia_test_mapping-2021.05.03-000002",
  "rolled_over" : true,
  "dry_run" : false,
  "conditions" : { }
}


Now

  • if I try to look at my new index mapping, I will find it empty.
  • in the field "zipcode" , I can put integer AND text
GET samia_test_mapping-2021.05.03-000002/_mapping

Response 

{
  "samia_test_mapping-2021.05.03-000002" : {
    "mappings" : { }
  }
}

POST  samia_test_mapping_alias/_doc 
{
  "firstname": "samia",
  "lastname": "mtimet",
  "zipcode": "78100"
}

POST  samia_test_mapping_alias/_doc 
{
  "firstname": "tania",
  "lastname": "mtimet",
  "zipcode": 78100
}

GET samia_test_mapping-2021.05.03-000002/_search
{
  "query": {
    "match_all": {}
  }
}

Response : 

{
  "took" : 958,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "samia_test_mapping-2021.05.03-000002",
        "_type" : "_doc",
        "_id" : "BgrTMnkB3bnl5NgYf6KJ",
        "_score" : 1.0,
        "_source" : {
          "firstname" : "samia",
          "lastname" : "mtimet",
          "zipcode" : "78100"
        }
      },
      {
        "_index" : "samia_test_mapping-2021.05.03-000002",
        "_type" : "_doc",
        "_id" : "VwrTMnkB3bnl5NgY66oN",
        "_score" : 1.0,
        "_source" : {
          "firstname" : "tania",
          "lastname" : "mtimet",
          "zipcode" : 78100
        }
      }
    ]
  }
}

Is it the normal behaviour ?

Hi @warkolm
You'll find my answer, after David's question.

Best regards,

These do not match due to the last underscore in the index_pattern field, and the index template is therefore not applied.

2 Likes

@Christian_Dahlqvist Thanks a lot ! My eyes must be replaced :slight_smile:

I'll correct it and test again !

It worked !

Sorry everybody.