All modules log into the same index in metricbeat - best practise?

Hi there

I remember the following blog post about index vs type discussion:

Especially one statement:

Do your documents have similar mappings? If no, use different indices.

IMHO, this applies to the different modules because only a few fields match with each other. Therefore, I'm wondering how does the current approach in metricbeat really scales in terms of memory/disk usage. Of course, I could update this within logstash and route it to different indices but then all dashboards are broken. What do you recommend?


You can already feed it now into different indices as the index name supports a format string. So you could use metricset.module for example as part of the index name.

In general, something we are actively discussing internally. Having one index per module can also lead to shard explosion.

About the dashboards: Having it under metricbeat-{}-* indices the dashboard pattern could still be metricbeat-* and this should still work.

I know what you mean which shards explosion. At least it might make sense - if it's not already set - to have one shard per index.

Haven't yet migrated to 5.x so I hope it works.


You are probably interested in this discussion:

This topic was automatically closed after 21 days. New replies are no longer allowed.