Template not working

Elastic Stack Elasticsearch
1

Hi Everyone ,

I have created a template for my indices with the intention that all indexes that fall under the index pattern will have same mapping.

This is my template
{
"sqe_template1": {
"order": 0,
"index_patterns": [
"sqe*",
"log-ecoa*"
],
"settings": {},
"mappings": {
"doc": {
"properties": {
"BytesReceived": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"BytesTransmitted": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"ClientTime": {
"type": "date",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Device": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"DeviceSerialNumber": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Environment": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"EnvironmentID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"EventOrder": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"KRPT": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Level": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Locale": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Location": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"ProcessId": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"ProdVer": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"ReceivedTime": {
"type": "date",
"fields": {
"keyword": {
"type": "keyword"
}
},
"format": "dd/MMM/yyyy:HH:mm:ss +SSSS"
},
"Role": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"SessionID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"SigID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Site": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Source": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Sponsor": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Status": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Study": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"StudyVer": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"SubjectID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"TransmissionID": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"Type": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"message": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
},
"aliases": {}
}
}

For testing purposes I added an extra field apart from the one's in the template and tried to POST it. Ideally I thought the extra field wont show up since I have put in a template.

But the extra field does get added. How should I force my data to follow the template

Kindly Help

2

Hi,

There's one things to take care about when you use template is the order of your actions.
1- set the template -> 2- create the index.
A template can't apply on an already existing index.
So as I understand you r problem you set a template and after posting a new doc with a new field this new field didn't have the mapping defined in the template?

Can you check the mapping of your index you must have the mapping defined in your template if it's not the case, you can always reindex in a new index to apply your template.

PS: if you put code in your message please use the format feature.

Look like your default mapping is

"XXX": {
  "type": "text",
  "fields": {
    "keyword": {
      "type": "keyword"
   }
}

you can use match_mapping_type to set the same mapping to all your string fields

{"string_fields" : {
                "match" : "*",
                "match_mapping_type": "string",
                "mapping" : {
                    "type": "text",
                    "fields": {
                      "keyword": {
                      "type": "keyword"
                    }
                }
            }}

You can put it at the end and it will catch all the string field and apply this mapping. It will reduce your template size.
If you add new string fields this mapping will apply, so you don't need to explicitly define your fields and update your index mapping.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.