Index name confusion with ILM enabled

Previously I used daily indexes defined like:

index: "filebeat-%{[agent.version]}-%{+yyyy.MM}"

Now I'm trying to use ILM so my daily indexes are managed more rationally. I have this config now:

setup.ilm.enabled: auto
setup.ilm.rollover_alias: "filebeat-%{[agent.version]}"
setup.ilm.pattern: "{now/M{yyyy-MM}}-000001"

But I'm getting a bunch of errors in the local log and no index is being created.

2019-07-16T21:00:54.509-0700	WARN	elasticsearch/client.go:781	Failed to create request parse http://cathedral.local:9200/_alias/filebeat-%{[agent.version]}: invalid URL escape "%{["`

So, okay, lets take the % escape trigger out...

    setup.ilm.rollover_alias: "filebeat-{[agent.version]}"

That eliminated the error but politely created an index named, literally "filebeat-{[agent.version]}". It didn't try to expand the version number, nor did it apparent the date & counter pattern to the end.

If I remove that entire variable chunk from the alias name, an index that follows the rules gets made "filebeat-2019-07-000001"

setup.ilm.rollover_alias: "filebeat"
setup.ilm.pattern: "{now/M{yyyy-MM}}-000001"

Maybe variables have to be in the pattern part?

setup.ilm.rollover_alias: "filebeat"
setup.ilm.pattern: "{[agent.version]}-{now/M{yyyy-MM}}-000001"

That just seems to drop the whole version part and just puts things in the filebeat-2019-07-000001 index again.

Hopefully there is something simple that I'm overlooking in frustration...

It appears that there is currently a bug related to changing the rollover alias:

However, since the default setting includes the agent version, I think you can get the name that you want simply by setting setup.ilm.pattern: "{now/M{yyyy-MM}}-000001" and omitting the rollover_alias setting altogether.

This will result in indices like filebeat-7.2.0-2019-07-000001

Okay, thanks for the link to that bug I'll track it, since the next issue was going to be 'I can't separate my logs by type in to different indices like I used to be able to do' and that's mentioned in that issue too.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.