Index name for modules

[tenney]

Using filebeat 7.3.0 on ubuntu and can't seem to change the index name for any modules that get enabled.

All I want to do is make the indexes that get created be done so monthly names instead of daily.

I would expect this to work, but has no change:

output.elasticsearch:
index: "filebeat-%{[agent.version]}-%{+yyyy.MM}"

Tried a lot a different things and it just seems to ignore them. Any ideas? Thanks.

[steffens (Steffen Siering)]

Please properly format logs and configs using the </> button. Filebeat configuration is sensitive to indentation. Without proper formatting it is difficult to see if there might be an error.

Do you have ILM enabled? Which exact index names are generated in Elasticsearch?

[tenney]

ILM is not enabled. Here is an example index name that gets generated:

filebeat-7.3.0-2019.08.13-000001

I can change the index name for non-module log consumption. I am using for example the nginx module by doing "filebeat modules enable nginx". I can't seem to change the index name for the nginx logs that now get consumed by filebeat.

It's default creates way to many indexes that is causing me performance issues with elasticsearch. I simply want to create a monthly index for it and would expect this to work:

output.elasticsearch:
  index: "filebeat-%{[agent.version]}-%{+yyyy.MM}"

[steffens (Steffen Siering)]

The default ILM mode is 'auto'. If beats detect the Elasticsearch cluster you send to has ILM support, then it will be enabled. The Index name filebeat-7.3.0-2019.08.13-000001 is created by ILM, not by beats. In fact beats uses a write alias named filebeat-%{[agent.version]} here.

With ILM enabled, the output.elasticsearch.index setting will be overwritten with the write alias.

In order to disable ILM stop beats, remove the filebeat-* templates, add setup.ilm.enabled: false to your config file and restart.