# Index template not applying to indices **URL:** \ **Category:** Elasticsearch\ **Created:** [November 26, 2018, 9:57pm UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244 "2018-11-26T21:57:58Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![alastairs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alastairs/32/38136_2.png) [@alastairs](https://discuss.elastic.co/u/alastairs)\ **Post date:** [November 26, 2018, 9:57pm UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/1 "2018-11-26T21:57:58Z")
Hi there We're using Filebeat 6.4.2 to scrape our logs from Kubernetes (with Docker) and are shipping our logs directly to Elastic Cloud, running 6.5.0. I loaded a custom index template into Elasticsearch to add a `.raw` field to our logs, as follows (this is a copy-paste from [Serilog's Elasticsearch sink](https://github.com/serilog/serilog-sinks-elasticsearch/)): ``` GET /_template/serilog-template { "serilog-template" : { "order" : 0, "index_patterns" : [ "filebeat-*" ], "settings" : { "index" : { "refresh_interval" : "5s" } }, "mappings" : { "_default_" : { "dynamic_templates" : [ { "numerics_in_fields" : { "path_match" : """fields\.[\d+]$""", "match_pattern" : "regex", "mapping" : { "type" : "text", "index" : true, "norms" : false } } }, { "string_fields" : { "match" : "*", "match_mapping_type" : "string", "mapping" : { "type" : "text", "index" : true, "norms" : false, "fields" : { "raw" : { "type" : "keyword", "index" : true, "ignore_above" : 256 } } } } } ], "properties" : { "message" : { "type" : "text", "index" : "true" }, "exceptions" : { "type" : "nested", "properties" : { "Depth" : { "type" : "integer" }, "RemoteStackIndex" : { "type" : "integer" }, "HResult" : { "type" : "integer" }, "StackTraceString" : { "type" : "text", "index" : "true" }, "RemoteStackTraceString" : { "type" : "text", "index" : "true" }, "ExceptionMessage" : { "type" : "object", "properties" : { "MemberType" : { "type" : "integer" } } } } } } } }, "aliases" : { } } } ``` In addition to this template, Filebeat has also installed its default template, which specifies `"order": 1`. I understand [from the docs](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/indices-templates.html#multiple-templates) that in this scenario the custom template will be applied before the Filebeat template. However, I do not see any of the `.raw` fields in my index mapping, or in the posted data. It seems almost as though the index pattern is being ignored entirely? We also have the same template applied to a custom index to which we submit log events directly (i.e., without Filebeat in the picture), and this _does_ include `.raw` fields in the index pattern and in the data, so my best guess is that the interplay with Filebeat is causing us problems here. ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 26, 2018, 10:05pm UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/2 "2018-11-26T22:05:09Z")
> [@alastairs](#): > > I understand [from the docs](https://www.elastic.co/guide/en/elasticsearch/reference/6.5/indices-templates.html#multiple-templates) that in this scenario the custom template will be applied before the Filebeat template. That is technically correct, but the important part of the docs here is "...with lower order being applied first and higher orders **overriding** them". ---
**Author:** ![alastairs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alastairs/32/38136_2.png) [@alastairs](https://discuss.elastic.co/u/alastairs)\ **Post date:** [November 27, 2018, 10:52am UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/3 "2018-11-27T10:52:08Z")
Thanks @Igor_Motov! So am I correct in thinking, then, that although my custom template is applied first, the Filebeat templates are overriding it to the extent that the `.raw` fields are being stripped out again? ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 27, 2018, 2:52pm UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/4 "2018-11-27T14:52:07Z")
> [@alastairs](#): > > So am I correct in thinking, then, that although my custom template is applied first, the Filebeat templates are overriding it to the extent that the `.raw` fields are being stripped out again? Once index is created, the templates have no effect on this index. So, the combined effect of all templates that were applied in the order that they were applied are in the mappings and settings of the index that was created. So, you need to take a look at the dynamic templates of one of the index that was created to see what the combined effect was. ---
**Author:** ![saif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saif/32/37431_2.png) [@saif](https://discuss.elastic.co/u/saif)\ **Post date:** [November 27, 2018, 2:58pm UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/5 "2018-11-27T14:58:45Z")
> [@alastairs](#): > > _default_ The default mapping is deprecated > **[default mapping | Elasticsearch Guide \[6.4\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/default-mapping.html)** ---
**Author:** ![alastairs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alastairs/32/38136_2.png) [@alastairs](https://discuss.elastic.co/u/alastairs)\ **Post date:** [November 27, 2018, 3:05pm UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/6 "2018-11-27T15:05:09Z")
Interesting, thanks @saif, that's good to know. AFAICT the `.raw` fields are appearing ok in the non-Filebeat indices, but it looks like that's something to update nonetheless. ---
**Author:** ![alastairs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alastairs/32/38136_2.png) [@alastairs](https://discuss.elastic.co/u/alastairs)\ **Post date:** [November 28, 2018, 8:50am UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/7 "2018-11-28T08:50:20Z")
Ok, I've updated the index template to have order 2 (so applies after Filebeat) and map to `doc` rather than `_default_`. I've used the [update by query API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html) to reindex the data in today's index, and I've refreshed the index pattern in Kibana. Here is (what I think is) the relevant portion of the dynamic templates applied to today's index: ```json { // Created by Filebeat index template "strings_as_keyword" : { "match_mapping_type" : "string", "mapping" : { "ignore_above" : 1024, "type" : "keyword" } } }, { // Created by custom index template "numerics_in_fields" : { "path_match" : """fields\.[\d+]$""", "match_pattern" : "regex", "mapping" : { "index" : true, "norms" : false, "type" : "text" } } }, { // Created by custom index template "string_fields" : { "match" : "*", "match_mapping_type" : "string", "mapping" : { "fields" : { "raw" : { "ignore_above" : 256, "index" : true, "type" : "keyword" } }, "index" : true, "norms" : false, "type" : "text" } } } ``` I feel as though there's a conflict here between the `strings_as_keyword` dynamic template from Filebeat and the `string_fields` dynamic template from my custom index template. (Needless to say, still no `.raw` fields in my index.) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 26, 2018, 8:50am UTC](https://discuss.elastic.co/t/index-template-not-applying-to-indices/158244/8 "2018-12-26T08:50:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.