Filebeat multiple index + ilm + index_template

Elastic Stack Beats
,
1

Hello,

In filebeat I use several indexes depending on the module.
I use the basic filebeat model based on the indexes to keep the field architecture.

  • filebeat-7.16.2-*
setup.template.enabled: true
setup.template.name: "filebeat-%{[agent.version]}"
setup.template.pattern: "filebeat-%{[agent.version]}-*"
setup.template.fields: "fields.yml"
setup.ilm.enabled: false

then I create indexes based on the module :

  • filebeat-7.16-2-haproxy-*
  • filebeat-7.16-2-logs-*
output.elasticsearch:
      indices:
        - index: "filebeat-ha-%{[agent.version]}-logs"
          when.equals:
            event.module: "system"
        - index: "filebeat-ha-%{[agent.version]}-haproxy"
          when.equals:
            event.module: "haproxy"

I have 2 templates:

  • filebeat-7.16.2 based on filebeat-7.16.2-* indexes → Mapping of filebeat (fields.yml)
  • filebeat-7.16-2-haproxy-* based on indexes filebeat-7.16.2-haproxy-*
{
  "index": {
    "lifecycle": {
      "name": "filebeat-custom",
      "rollover_alias": "filebeat-7.16.2-haproxy"
    }
  }
}
  • filebeat-7.16-2-log-* based on filebeat-7.16.2-logs-* indexes
{
  "index": {
    "lifecycle": {
      "name": "filebeat-custom",
      "rollover_alias": "filebeat-7.16.2-logs"
    }
  }
}

With legacy indexes and ordering, this works, when creating the :

  • it takes the fields from filebeat-7.16.2
  • then the ilm settings of filebeat-7.16.2-haproxy

How do you do it with template_index?
The order doesn't work anymore, and you have to use the template components and I don't see how to do it.

reading the docs, i can't find the information to combine index splitting on filebeat with index_template.

2 Likes
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.