# Is it mandatory to load the index template into Elasticsearch manually when using filebeat?

**URL:** <https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 4, 2018, 5:22pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668 "2018-05-04T17:22:19Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\
**Post date:** [May 4, 2018, 5:22pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668/1 "2018-05-04T17:22:19Z")

</div>

Hi,

I am pretty new to Filebeat. I am using Logstash to send logs to another Logstash which will send logs to Elasticsearch(Logstash1 to Logstash2 to Elasticsearch). Now I am planning to replace Logstash1 with Filebeat. Documents say to load the index template into Elasticsearch manually and my question is - if I am already using a template in Elasticsearch, do I still need to load the index template into Elasticsearch manually?

Thanks

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 4, 2018, 7:17pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668/2 "2018-05-04T19:17:25Z")

</div>

It depends on where you are writing the Filebeat data to and whether the index template you have contains the appropriate mappings for the Filebeat fields. To be safe I would follow the directions.

My recommendation is to install the index template provided by Filebeat and write your data into the prescribed `filebeat-<version>-*` indices. Basically following the documentation and using the defaults. So you would manually load the Filebeat template as per the [instructions](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html#load-template-manually). This template will apply to indices matching `filebeat-6.2.4-*`.

Then configure Logstash to output the beat data with this [config](https://www.elastic.co/guide/en/beats/libbeat/6.2/logstash-installation.html#logstash-setup).

```auto
output {
  elasticsearch {
    hosts => "localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" 
    document_type => "%{[@metadata][type]}" 
  }
}

```

If you make any adjustments to index naming pattern then the template also needs to be changed.

---

<div class="post-metadata">

**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\
**Post date:** [May 4, 2018, 8:11pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668/3 "2018-05-04T20:11:01Z")

</div>

Hi Andrew,

As per your suggestion, I would install the index template. But I have 4 Elasticsearch nodes(one coordinating node and 3 Data nodes). Do I need to install template on all 3 Data nodes one at a time using

`curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/filebeat-6.2.4 -d@filebeat.template.json`

And also is there a way to drop Beat fields? I am basically looking for same exact fields what I have right now with Logstash.

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 4, 2018, 9:05pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668/4 "2018-05-04T21:05:31Z")

</div>

The template becomes part of the cluster state so you can add it to any one node and it will sync to the others.

You can drop fields on the Filebeat side with a [drop\_fields](https://www.elastic.co/guide/en/beats/filebeat/6.2/drop-fields.html) [processor](https://www.elastic.co/guide/en/beats/filebeat/6.2/filtering-and-enhancing-data.html).

```auto
processors:
 - drop_fields:
     fields: [field1, field2]

```

---

<div class="post-metadata">

**Author:** ![amruth](https://avatars.discourse-cdn.com/v4/letter/a/43a26b/32.png) [@amruth](https://discuss.elastic.co/u/amruth)\
**Post date:** [May 18, 2018, 5:08pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668/5 "2018-05-18T17:08:58Z")

</div>

Thanks Adrew.

I have another question - I am ingesting logs to an index called smaple-%{+YYYY.MM} and I don't have template in elasticsearch and am planning to have one template by the time it creates next index(i.e next month).

Since I am planning to replace Logstash1 with Filebeat, I will have to load the template manually. How should I create a template in elasticsearch for next month?

I am confused between Filebeat template and the one we create in ES using [Index Template](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html)

Thanks

---

<div class="post-metadata">

**Author:** ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)\
**Post date:** [May 18, 2018, 5:54pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668/6 "2018-05-18T17:54:21Z")

</div>

You can let Filebeat generate an index template for your version of Elasticsearch. Then you can manually install it. Between those steps you can customize the index template as needed (like customize number of shards or add additional fields).

```auto
# Export template to file.
filebeat export template --es.version=6.2.4 -E setup.template.pattern="smaple-*" > filebeat.template.json

# Load template to ES.
curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/smaple @filebeat.template.json

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 15, 2018, 5:54pm UTC](https://discuss.elastic.co/t/is-it-mandatory-to-load-the-index-template-into-elasticsearch-manually-when-using-filebeat/130668/7 "2018-06-15T17:54:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.