Setting up multiple ILM policies on a filebeat with multiple indices as its output

Elastic Stack Beats
1

I've been trying to set up ILM policies on a filebeat which has a module enabled for additional data source to process. So, I'll need to apply two different policies to two different indexes on the same filebeat.

ILM policy names:

  1. filebeat-7.4.2 >> this is the native one used by default by a filebeat.

  2. vpc_flow-policy >> a custom policy for flow logs.

    filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/commands.log

filebeat.config.modules:
  path: /etc/filebeat/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 1
  index.codec: best_compression

tags: ["flow_logs"]

output.elasticsearch:
  indices:
    - index: "vpc_flow-%{+yyyy.MM.dd}"
      when.contains:
        event.type: "flow"

processors:
    - add_host_metadata:
       netinfo.enabled: true
    - add_cloud_metadata: ~

xpack.monitoring.enabled: true

Where should I add ILM configurations which would only apply to vpc_flow-* indexes and not filebeat-* ones? I already created ILM in Kibana.

--Thanks

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.