# All modules log into the same index in metricbeat - best practise?

**URL:** <https://discuss.elastic.co/t/all-modules-log-into-the-same-index-in-metricbeat-best-practise/80192>\
**Category:** Beats\
**Created:** [March 27, 2017, 7:06pm UTC](https://discuss.elastic.co/t/all-modules-log-into-the-same-index-in-metricbeat-best-practise/80192 "2017-03-27T19:06:36Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![owulff](https://avatars.discourse-cdn.com/v4/letter/o/f475e1/32.png) [@owulff](https://discuss.elastic.co/u/owulff)\
**Post date:** [March 27, 2017, 7:06pm UTC](https://discuss.elastic.co/t/all-modules-log-into-the-same-index-in-metricbeat-best-practise/80192/1 "2017-03-27T19:06:36Z")

</div>

Hi there

I remember the following blog post about index vs type discussion:

> **[Index vs. Type](https://www.elastic.co/blog/index-vs-type)**

Especially one statement:

> 

Do your documents have similar mappings? If no, use different indices.

> 

IMHO, this applies to the different modules because only a few fields match with each other. Therefore, I'm wondering how does the current approach in metricbeat really scales in terms of memory/disk usage. Of course, I could update this within logstash and route it to different indices but then all dashboards are broken. What do you recommend?

Thanks  
Oli

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 28, 2017, 2:33pm UTC](https://discuss.elastic.co/t/all-modules-log-into-the-same-index-in-metricbeat-best-practise/80192/2 "2017-03-28T14:33:03Z")

</div>

You can already feed it now into different indices as the index name supports a format string. So you could use `metricset.module` for example as part of the index name. [https://www.elastic.co/guide/en/beats/libbeat/current/config-file-format-type.html#\_format\_string\_sprintf](https://www.elastic.co/guide/en/beats/libbeat/current/config-file-format-type.html#_format_string_sprintf)

In general, something we are actively discussing internally. Having one index per module can also lead to shard explosion.

About the dashboards: Having it under `metricbeat-{module.name}-*` indices the dashboard pattern could still be `metricbeat-*` and this should still work.

---

<div class="post-metadata">

**Author:** ![owulff](https://avatars.discourse-cdn.com/v4/letter/o/f475e1/32.png) [@owulff](https://discuss.elastic.co/u/owulff)\
**Post date:** [March 29, 2017, 8:29pm UTC](https://discuss.elastic.co/t/all-modules-log-into-the-same-index-in-metricbeat-best-practise/80192/3 "2017-03-29T20:29:17Z")

</div>

I know what you mean which shards explosion. At least it might make sense - if it's not already set - to have one shard per index.

Haven't yet migrated to 5.x so I hope it works.

Thanks  
Oli

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [March 30, 2017, 12:37pm UTC](https://discuss.elastic.co/t/all-modules-log-into-the-same-index-in-metricbeat-best-practise/80192/4 "2017-03-30T12:37:23Z")

</div>

You are probably interested in this discussion: [https://github.com/elastic/beats/issues/3431](https://github.com/elastic/beats/issues/3431)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2017, 7:06pm UTC](https://discuss.elastic.co/t/all-modules-log-into-the-same-index-in-metricbeat-best-practise/80192/5 "2017-04-17T19:06:52Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.