Examples for setting up Beats (modules) > Logstash > ES with ILM?

Are there any basic configuration examples showing how to set up a Beats (with modules and dashboards) > Logstash > ES pipeline using Index Lifecycle Management? Currently I'm familiar with the "classic" setup with Metricbeat and Filebeat pushing logs to logstash. Logstash is configured to create daily indices with %{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd} for the name.

To set this up to use ILM, I need to run the beats setup commands with ilm enabled to create the templates, which creates the policy and index template (something like filebeat-7.0.1-2019-05-06-000001) That is the "real" index, but logstash should just be configured to write to the alias filebeat-7.0.1?

Or is logstash supposed to be configured to use ILM and I need to set up the template and alias myself? Should it still separate indices per beat or just use one logstash index? Would that break the built-in dashboards? If anyone has a basic working example for a setup like this I would really appreciate it!

I have the same question. I have everything working but ILM is having issues. image

Has anyone figured this out yet? I'm facing the same issue.

Do you have logstash configured for daily indexes? Something like index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"?

When you set up ILM it creates an alias like filebeat-7.0.1 and ILM handles rolling over and creating real indexes. So if you configure logstash to just push to the alias index => "%{[@metadata][beat]}-%{[@metadata][version]}" it should get rid of those errors.

The problem I ran in to was it all seems to work as long as you never touch it again. If you ever delete indexes manually, I still havent figured out how to configure it again to restore it to a working state.

yup I have logstash creating daily indexes. Alright so I should make that change. Now I am wondering if this will be broken for me now though because I just reindexed and deleted an index.

Now it's creating the index but no ILM policy or alias. Let me start from scratch again.

I started from scratch completely and es is not creating ilm at all using index => "%{[@metadata][beat]}-%{[@metadata][version]}"

i think ILM configuration is created when you run the beats setup commands like filebeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'.

I would stop the logstash service before cleaning up indexes so it doesnt create a new one before you have a chance to run setup

Thanks for the help Pete. This is driving me nuts. The documentation states this should all be automagic but it's just not quite meshing together correctly. Luckily I have the time to invest right now.

That seems to be working, thank you. The initial index is still created with a date stamp in the name, but it appears to be working.