# Metricbeat pushing the whole template and the induced sparsity

**URL:** https://discuss.elastic.co/t/metricbeat-pushing-the-whole-template-and-the-induced-sparsity/168545
**Category:** Beats
**Tags:** metricbeat
**Created:** [February 15, 2019, 8:15am UTC](https://discuss.elastic.co/t/metricbeat-pushing-the-whole-template-and-the-induced-sparsity/168545 "2019-02-15T08:15:40Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![wokmichel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wokmichel/32/48057_2.png) [@wokmichel](https://discuss.elastic.co/u/wokmichel)
#### Post date: [February 15, 2019, 8:15am UTC](https://discuss.elastic.co/t/metricbeat-pushing-the-whole-template-and-the-induced-sparsity/168545/1 "2019-02-15T08:15:40Z")

</div>

Hello,  
a question about metricbeat template and sparsity :

My understanding is that when setting up the metricbeat template in ElasticSearch via the following command :

`metricbeat setup --template`

The whole template is loaded into ElasticSearch according to what is described into fields.yml, regardless of what modules are enabled. That is to say, for instance, that, if I only want to use the system module, every document in elasticsearch will have a total of approximately 2500 fields with a vas majority of those being empty.

Although I am aware that sparsity handling has been greatly optimized and improved since the introduction of Lucene 7, I was wondering if there was something I was missing when it comes to setting up the template only for the enabled modules or if this was meant to be used this way, with the whole template loaded.

**In a nutshell :**  
Is it a problem (considering the incuced sparsity) to have the whole, huge metricbeat template loaded in ElasticSearch or is it normal behavior ?  
If not, is there a way (other than manually editing the fields.yml) to choose which part of the template to push to ElasticSearch, according to the enabled modules ?

Thanks a lot 🙂

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [February 21, 2019, 5:12am UTC](https://discuss.elastic.co/t/metricbeat-pushing-the-whole-template-and-the-induced-sparsity/168545/2 "2019-02-21T05:12:46Z")

</div>

Hi @wokmichel great question 🙂 I usually don't even run `metricbeat setup --template` command since Metricbeat automatically loads the template file `fields.yml` anyway. I guess one thing I can think of is to modify the `fields.yml` to only keep the fields that you care about so you don't load all the 2500 fields.

---

<div class="post-metadata">

### Author: ![wokmichel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wokmichel/32/48057_2.png) [@wokmichel](https://discuss.elastic.co/u/wokmichel)
#### Post date: [February 24, 2019, 8:33pm UTC](https://discuss.elastic.co/t/metricbeat-pushing-the-whole-template-and-the-induced-sparsity/168545/3 "2019-02-24T20:33:03Z")

</div>

Hello there and thanks for your answer.  
As I understand from your answer, this is normal that the template is so huge and that it induces sparsity ?

Thanks

---

<div class="post-metadata">

### Author: ![Kaiyan\_Sheng](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kaiyan_sheng/32/38247_2.png) [@Kaiyan\_Sheng](https://discuss.elastic.co/u/Kaiyan_Sheng)
#### Post date: [March 3, 2019, 10:49pm UTC](https://discuss.elastic.co/t/metricbeat-pushing-the-whole-template-and-the-induced-sparsity/168545/4 "2019-03-03T22:49:27Z")

</div>

Yeah my understanding is, `metricbeat setup --template` loads everything from fields.yml. And this fields.yml is a combination of all fields.yml from all modules in metricbeat. So if want a smaller fields.yml, you can remove fields.yml under specific modules and then run `make update` to rebuild metricbeat.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 31, 2019, 10:49pm UTC](https://discuss.elastic.co/t/metricbeat-pushing-the-whole-template-and-the-induced-sparsity/168545/5 "2019-03-31T22:49:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
