Index name confusion with ILM enabled

[DPattee]

Previously I used daily indexes defined like:

output.elasticsearch:
index: "filebeat-%{[agent.version]}-%{+yyyy.MM}"

Now I'm trying to use ILM so my daily indexes are managed more rationally. I have this config now:

setup.ilm.enabled: auto
setup.ilm.rollover_alias: "filebeat-%{[agent.version]}"
setup.ilm.pattern: "{now/M{yyyy-MM}}-000001"

But I'm getting a bunch of errors in the local log and no index is being created.

2019-07-16T21:00:54.509-0700	WARN	elasticsearch/client.go:781	Failed to create request parse http://cathedral.local:9200/_alias/filebeat-%{[agent.version]}: invalid URL escape "%{["`

So, okay, lets take the % escape trigger out...

    setup.ilm.rollover_alias: "filebeat-{[agent.version]}"

That eliminated the error but politely created an index named, literally "filebeat-{[agent.version]}". It didn't try to expand the version number, nor did it apparent the date & counter pattern to the end.

If I remove that entire variable chunk from the alias name, an index that follows the rules gets made "filebeat-2019-07-000001"

setup.ilm.rollover_alias: "filebeat"
setup.ilm.pattern: "{now/M{yyyy-MM}}-000001"

Maybe variables have to be in the pattern part?

setup.ilm.rollover_alias: "filebeat"
setup.ilm.pattern: "{[agent.version]}-{now/M{yyyy-MM}}-000001"

That just seems to drop the whole version part and just puts things in the filebeat-2019-07-000001 index again.

Hopefully there is something simple that I'm overlooking in frustration...

[dedemorton]

It appears that there is currently a bug related to changing the rollover alias: https://github.com/elastic/beats/issues/12233

However, since the default setting includes the agent version, I think you can get the name that you want simply by setting setup.ilm.pattern: "{now/M{yyyy-MM}}-000001" and omitting the rollover_alias setting altogether.

This will result in indices like filebeat-7.2.0-2019-07-000001

[DPattee]

Okay, thanks for the link to that bug I'll track it, since the next issue was going to be 'I can't separate my logs by type in to different indices like I used to be able to do' and that's mentioned in that issue too.