# Initial startup (bulk api) does not use my template

**URL:** <https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318>\
**Category:** Logstash\
**Created:** [May 30, 2016, 12:31pm UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318 "2016-05-30T12:31:17Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![remco](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@remco](https://discuss.elastic.co/u/remco)\
**Post date:** [May 30, 2016, 12:31pm UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/1 "2016-05-30T12:31:17Z")

</div>

Hi,

I have installed ELK (eleasticsearch 2.3.1 logstash 2.3.0 and kibana 4.5). when the system is started and ELK is started for the first time (via linux services).  
Logstash mappings are not created with my own template (missing e.g. the .raw files).

In the elastics search logging i get the following logging:

```auto
[2016-05-30 10:07:01,794][INFO][cluster.metadata] [node-1] [my-logstash-2016.05.28] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [sysmon_x]
[2016-05-30 10:07:01,921][INFO][cluster.metadata] [node-1] [my-logstash-2016.05.29] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [messages]

```

Reason i use this template because sometime the raw fields are not created at startup.  
I want to use my own template to have more control over the fields (which will have .raw fields and which don't).

**My Question is, why is my template not used?**

```auto
---------- EXTRA INFO BELOW ----------

```

Logstash output.conf:

```auto
output{
        if "_grokparsefailure" in [tags]{
                #stdout { codec => rubydebug }
                file {
                        codec => rubydebug
                        path => "/tmp/logstash.log"
                }
        }
        elasticsearch { 
                hosts => "my-1:13076"
                index => "my-logstash-%{+YYYY.MM.dd}"
                template => "/etc/logstash/template/elasticsearch-template.json"
                template_name => "my-logstash*"
                template_overwrite => "true"
                manage_template => "false"
                flush_size => 100
                workers => 1
        }
}

```

My template:  
90% same as default.  
different all enabled = false  
and removed ignore above 256 for raw field

```auto
{
  "template" : "my-logstash*",
  "settings" : {
    "index.refresh_interval" : "5s"
  },
  "mappings" : {
    "_default_" : {
      "_all" : {"enabled" : false, "omit_norms" : true},
      "dynamic_templates" : [ {
        "message_field" : {
          "match" : "message",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "string", "index" : "analyzed", "omit_norms" : true,
            "fielddata" : { "format" : "disabled" }
          }
        }
      }, {
        "string_fields" : {
          "match" : "*",
          "match_mapping_type" : "string",
          "mapping" : {
            "type" : "string", "index" : "analyzed", "omit_norms" : true,
            "fielddata" : { "format" : "disabled" },
            "fields" : {
              "raw" : {"type": "string", "index" : "not_analyzed", "doc_values" : true}
            }
          }
        }
      }, {
        "float_fields" : {
          "match" : "*",
          "match_mapping_type" : "float",
          "mapping" : { "type" : "float", "doc_values" : true }
        }
      }, {
        "double_fields" : {
          "match" : "*",
          "match_mapping_type" : "double",
          "mapping" : { "type" : "double", "doc_values" : true }
        }
      }, {
        "byte_fields" : {
          "match" : "*",
          "match_mapping_type" : "byte",
          "mapping" : { "type" : "byte", "doc_values" : true }
        }
      }, {
        "short_fields" : {
          "match" : "*",
          "match_mapping_type" : "short",
          "mapping" : { "type" : "short", "doc_values" : true }
        }
      }, {
        "integer_fields" : {
          "match" : "*",
          "match_mapping_type" : "integer",
          "mapping" : { "type" : "integer", "doc_values" : true }
        }
      }, {
        "long_fields" : {
          "match" : "*",
          "match_mapping_type" : "long",
          "mapping" : { "type" : "long", "doc_values" : true }
        }
      }, {
        "date_fields" : {
          "match" : "*",
          "match_mapping_type" : "date",
          "mapping" : { "type" : "date", "doc_values" : true }
        }
      }, {
        "geo_point_fields" : {
          "match" : "*",
          "match_mapping_type" : "geo_point",
          "mapping" : { "type" : "geo_point", "doc_values" : true }
        }
      } ],
      "properties" : {
        "@timestamp": { "type": "date", "doc_values" : true },
        "@version": { "type": "string", "index": "not_analyzed", "doc_values" : true },
        "geoip" : {
          "type" : "object",
          "dynamic": true,
          "properties" : {
            "ip": { "type": "ip", "doc_values" : true },
            "location" : { "type" : "geo_point", "doc_values" : true },
            "latitude" : { "type" : "float", "doc_values" : true },
            "longitude" : { "type" : "float", "doc_values" : true }
          }
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![remco](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@remco](https://discuss.elastic.co/u/remco)\
**Post date:** [May 30, 2016, 2:21pm UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/2 "2016-05-30T14:21:08Z")

</div>

Sometimes i get the following (might it have todo with starting up sequence?):

```auto
[2016-05-30 12:57:41,722][INFO][cluster.metadata] [node-1] [my-logstash-2016.05.30] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [karaf, ospl, asl, sysmon_x]
[2016-05-30 12:57:42,210][INFO][cluster.metadata] [node-1] [my-logstash-2016.05.29] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [messages]
[2016-05-30 12:57:42,652][INFO][cluster.metadata] [node-1] [my-logstash-2016.05.26] creating index, cause [auto(bulk api)], templates [my-logstash*], shards [5]/[1], mappings [_default_, local3]
[2016-05-30 12:57:42,728][INFO][cluster.metadata] [node-1] [my-logstash-2016.05.28] creating index, cause [auto(bulk api)], templates [my-logstash*], shards [5]/[1], mappings [_default_, sysmon_x]

```

this is executed with the following output.conf of logstash:

```auto
output{
        if "_grokparsefailure" in [tags]{
                #stdout { codec => rubydebug }
                file {
                        codec => rubydebug
                        path => "/tmp/logstash.log"
                }
        }
        elasticsearch { 
                hosts => "my-1:13076"
                index => "my-logstash-%{+YYYY.MM.dd}"
                template => "/etc/logstash/template/elasticsearch-template.json"
                template_name => "my-logstash*"
                template_overwrite => "true"
                flush_size => 100
                workers => 1
        }
}

```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 31, 2016, 1:34am UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/3 "2016-05-31T01:34:01Z")

</div>

Do you have other templates in ES?

---

<div class="post-metadata">

**Author:** ![remco](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@remco](https://discuss.elastic.co/u/remco)\
**Post date:** [May 31, 2016, 4:16am UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/4 "2016-05-31T04:16:42Z")

</div>

Hi mark,

No i don't have any other costum templates in es. Only the default es might use for example for kibana.

---

<div class="post-metadata">

**Author:** ![remco](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@remco](https://discuss.elastic.co/u/remco)\
**Post date:** [June 15, 2016, 7:17am UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/5 "2016-06-15T07:17:43Z")

</div>

After some testing i see:

- When i start Elasticsearch, Logstah, Kibana manual then reboot the system =\> OK
- When i reboot first (so first time start is done by the system) =\> NOK

Both situation it is started by root.

any suggestions?

---

<div class="post-metadata">

**Author:** ![remco](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@remco](https://discuss.elastic.co/u/remco)\
**Post date:** [July 4, 2016, 5:21am UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/6 "2016-07-04T05:21:12Z")

</div>

Hi,

I see the same result now for topbeat also. Some times the template is not used to created the mappings (at first start).

Could help overwrite template setting?  
Or doesn't that work when the default template is used?

Or is there a way to inject the templates / mappings manually?

---

<div class="post-metadata">

**Author:** ![zhangchengshan](https://avatars.discourse-cdn.com/v4/letter/z/3d9bf3/32.png) [@zhangchengshan](https://discuss.elastic.co/u/zhangchengshan)\
**Post date:** [May 17, 2017, 11:15am UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/7 "2017-05-17T11:15:37Z")

</div>

Hi, i got the same problem, do you have solved it?

---

<div class="post-metadata">

**Author:** ![remco](https://avatars.discourse-cdn.com/v4/letter/r/9d8465/32.png) [@remco](https://discuss.elastic.co/u/remco)\
**Post date:** [May 17, 2017, 3:03pm UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/8 "2017-05-17T15:03:17Z")

</div>

No I still have the issue. For me the workaround works.  
Workaround: start logstash/topbeat/metricbeat/kibana on the system where elasicsearch is running first. After that start on the other systems logstash/topbeat/metricbeat.

Hope it helps.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:26am UTC](https://discuss.elastic.co/t/initial-startup-bulk-api-does-not-use-my-template/51318/9 "2017-07-06T04:26:31Z")

</div>