Seeking clarification on how 'filebeat setup' works with rollover_alias defined, etc

I am configuring filebeat to send various module output to Elastic Search v7. We are using ILM. I am looking at the ILM settings here: Configure index lifecycle management - ES 7 and I am wondering if someone could clarify a few items. Here is what I do know:

  • I know when I run the command filebeat setup It loads dashboards, templates, etc. Among these, in the template, it sets an index pattern:
index_patterns" : [
      "filebeat-7.6.2-*"
    ],

and roll-over alias

lifecycle" : {
          "name" : "filebeat",
          "rollover_alias" : "filebeat-7.6.2"
        },
  • I know Kibana's Dashboards are, in some way configured to look at this default index pattern when it searches for fields. Right? I'm not sure how this particular point works.

  • I know when I update File beat it automatically re-loads this information.

  • I know, since I am using ILM, the index option in Elasticsearch.output is not available. setup.template.name and setup.template.pattern are also unavailable.

For a subset of machines, I want to place their data in a different index with a different ILM. The mappings,ingest pipelines, and dashboards for the filebeat modules I use will be the same, I just want to re-route them into a separate index so I can apply a different ILM. I'm stuck on how tobest do this. Looking at the ILM documentation, I can see there is an setup.ilm.rollover_alias option. My questions are:

  • If I define the setup.ilm.rollover_alias, will this separate the subset of machines into a different index as well?

  • Assuming it does, How does this interact with filebeat setup, the Kibana dashboards, and template loading? Will this change be auto-applied every time I update filebeat?

  • Assuming it does not, how can I split these up, still use the filebeat & filebeat modules stuff, and ILM?

  • Lastly, if I use the directive setup.ilm.policy_name is this telling the template to use a per-existing ilm policy in elastic search, or is this telling filebeat to name the policy it uploads into elastic search this name?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.