If enable ILM in config file, then cannot set custom elastic output index

Elastic Stack Beats
,
1

setup.template.overwrite: true
output.elasticsearch.index: "winlogbeat=%{[agent.version]}"-city-firm-hostname-%{+yyyy,MM,dd}-000001"
setup.template.name: "winlogbeat"
setup.template.pattern: "winlogbeat-*"
setup.ilm.overwrite: true
setup.ilm.enabled: true << ----- must be set to false to generate custom output.elasticsearch.index names?.

If I enable ILM per "setup.ilm.enable: true", then the output index is not generated on the stipulated name format "winlogbeat=%{[agent.version]}"-city-firm-hostname-%{+yyyy,MM,dd}-000001".

If setup.ilm.enabled: false, then there is no problem with the output index stipulated name format.

Thanks in advance

2

Hello Tony,

with setup.ilm.enabled: false Elastic uses the config provided in output.elasticsearch.index. If you enable setup.ilm.enabled, Elastic starts using setup.ilm.rollover_alias and setup.ilm.pattern.
So, to achieve what you want you have to update your configuration(untested):

setup.ilm.rollover_alias: "winlogbeat-%{[agent.version]}-city-firm-hostname"
setup.ilm.pattern: "{now{+yyyy,MM,dd}}-000001"

Best regards
Wolfram

4

Hi Wolfram,
IIlegal_argument_exception: index.lifecycle.rollover_alias [winlogbeat-7.9.2] does not point to index [winlogbeat-7.9.2-city-firm-hostname10-2020.10.08-000001] was rendered in Index Management.
Some say I should not define Aliases in index templates which I have not. This error has occurred in 16 indices but not all. There are question with no solution in the listserv that I can see thus far.

Thanks in advance

5

Hello Tony,

The old rollover alias winlogbeat-7.9.2 was created using the old template and therefore collides with the new entries. I think you can delete the old rollover alias using the alias API as no new data should be coing to this alias after you have changed your Beats config.

Best regards
Wolfram

6

Thanks Wofram, Your recommendation worked for the first time, then something caused Elastic not picking up the index after an 15mins. When I reset my config. It sends me another error
Exiting: resource 'winlogbeat-7.9.2' exists, but it is not an alias .
Sorry for the series of issues on ilm which is causing me havoc.

7

a bit more clarification, it should be 'winlogbeat-7.9.2-city-firm-host' exists, but it is not alias.

8

Thanks Wolfram, got it working after deleting the old indices as per advice and to ensure manually creating index alias through APIs too . Also manually creating the _rollover index alias to resolve the interim error messages (don't know why the ilm GUI was not sufficient).

1 Like
9

Thanks Wolfram, corrected the setup.ilm.pattern: "{now{+yyyy,MM,dd}}-000001" to setup.ilm.pattern: "{now/M{YYYY.MM.dd}}-000001" in the winlogbeat.yml, which produced the right rollover_alias "winlogbeat-7.9.2-city-firm-hostname10" appended with YYYY.MM.dd-000001

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.