Generate automatic index names like "%{[container.name]}-%{+yyyy.MM.dd}" with ILM enabled

Elastic Stack Beats
,
1

Hello, our worked config of filebeat.yml:

#===== Filebeat Inputs ======
filebeat.inputs:
- type: docker
  combine_partial: true
  containers:
    path: "/var/lib/docker/containers"
    ids :
      - "*"

#===== Filebeat modules =====
filebeat.config.modules:
    path: "${path.config}/modules.d/*.yml"
    reload.enabled: false

#===== Elasticsearch template setting, we do not create any special template for new containers =====
setup.template:
  name: '%{[container.name]}'
  pattern: '%{[container.name]}-*'
  enabled: false

setup.ilm.enabled: false

#===== Elastic Cloud ======
cloud.id  : "ID"
cloud.auth: "AUTH"

#===== Outputs, any new container adds automatic =====
output.elasticsearch.index: "%{[container.name]}-%{+yyyy.MM.dd}"

#==== Processors =====
processors:
    - add_fields:
        target: ''
        fields:
          servertype: "some-servertype"
          domain: "some-domain"
    - add_host_metadata  : ~
    - add_cloud_metadata : ~
    - add_docker_metadata: ~
    - decode_json_fields:
        fields: ['message']
        target: json
        max_depth: 4

When we enable ILM, filebeat can not generate automatic index names "%{[container.name]}-%{+yyyy.MM.dd}" ..
How fix this with ILM enabled? Thank for any recommendation..

2

When ILM creates the first index, it does so with a timestamp. The timestamp in the index name will not change as it's the date that the policy was first used, not the date that the index was rolled.

1 Like
3

If index names will be like

"%{[container.name]}-%{+yyyy.MM.dd}-\\d+"

this does not problem.
Could you give sample of config please?

4

Если задействовать ILM (setup.ilm.enabled: true), то параметры setup.template.name и setup.template.pattern игнорируются.
На мой взгляд ILM отлично задействовать для существующих индексов (ротация, слияние, замозозка, удаление...). Новый индекс приходится создавать через API создания индексов, естественно с учётом подстановок

1 Like
5

I don't understand your question sorry.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.