Index Template getting overwritten on when indexing first document

Elastic Stack Elasticsearch
1

I'm seeing a strange issue with my index templates in elasticsearch where Elasticsearch is overwriting the index templates I have defined with it's default mappings. I have a index template for all of my rpt_* indicies. When i do a GET /_template I see.

{
"reporting_template_msghist":{
"order":0,
"template":"rpt_*",
"settings":{

  },
  "mappings":{  
     "msgHistory":{  
        "properties":{  
           "date":{  
              "format":"dateOptionalTime",
              "type":"date"
           },
           "campaignId":{  
              "index":"not_analyzed",
              "type":"string"
           },
           "venueId":{  
              "index":"not_analyzed",
              "type":"string"
           },
           "state":{  
              "index":"not_analyzed",
              "type":"string"
           }
        }
     }
  },
  "aliases":{  

  }

}
}

I then create a new document (along with a new index) in the index rpt_201506 with body:

{
  "campaignId": "abc-123-zxyz",
  "date": "2015-01-23T19:28:07.000+0000",
  "venueId": "venue_id",
  "state": "read"
}

When I investigate the index mappings after the index and first document is created they are now:

"msgHistory":{  
  "properties":{  
               "date":{  
                  "format":"dateOptionalTime",
                  "type":"date"
               },
               "campaignId":{  
                  "type":"string"
               },
               "venueId":{  
                  "type":"string"
               },
               "state":{  
                  "type":"string"
               }
  }
}

Hopefully, I'm just doing something stupid, but I didn't expect Elasticsearch to try and dynamically determine the mapping since I already had the index template mapping defined.

2

How are you inserting these?

Template is honoured in my repro

    PUT _template/reporting_template_msghist
{
  "order": 0,
  "template": "rpt_*",
  "settings": {},
  "mappings": {
    "msgHistory": {
      "properties": {
        "date": {
          "format": "dateOptionalTime",
          "type": "date"
        },
        "campaignId": {
          "index": "not_analyzed",
          "type": "string"
        },
        "venueId": {
          "index": "not_analyzed",
          "type": "string"
        },
        "state": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    }
  },
  "aliases": {}
}

PUT rpt_test/msgHistory/1
{
  "campaignId": "abc-123-zxyz",
  "date": "2015-01-23T19:28:07.000+0000",
  "venueId": "venue_id",
  "state": "read"
}

GET rpt_test/_mapping

returns

{
   "rpt_test": {
      "mappings": {
         "msgHistory": {
            "properties": {
               "campaignId": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "date": {
                  "type": "date",
                  "format": "dateOptionalTime"
               },
               "state": {
                  "type": "string",
                  "index": "not_analyzed"
               },
               "venueId": {
                  "type": "string",
                  "index": "not_analyzed"
               }
            }
         }
      }
   }
}
3

Are you setting the msgHistory type for your document when you index?

4

@nellicus I'm using the java client to create the index template and document. I'm currently retrying using just the REST API directly.

5

@warkolm I'm pretty sure I am setting the type, but I will double check.

6

Thanks for the help. I am a little further along. It looks like there is something wrong the way I'm adding the index template through the java API. I have copied a sample of the code I am using:

XContentBuilder json = jsonBuilder()
                .startObject()
                .startObject("properties")
                    .startObject("campaignId")
                        .field("type", "string")
                        .field("index", "not_analyzed")
                    .endObject()
                    .startObject("date")
                        .field("type", "date")
                        .field("format", "dateOptionalTime")
                    .endObject()
                    .startObject("state")
                        .field("type", "string")
                        .field("index", "not_analyzed")
                    .endObject()
                    .startObject("venueId")
                        .field("type", "string")
                        .field("index", "not_analyzed")
                    .endObject()
                .endObject()
            .endObject();

client.admin().indices().preparePutTemplate("reporting_template_msghist")
                    .setTemplate("rpt_*")
                    .addMapping("msgHistory", json)
                    .execute()
                    .actionGet();
7

I have tracked down my problem to an issue in the Java API (i think) - https://github.com/elastic/elasticsearch/issues/10436

Thanks for the help.