Filebeatt doesn't create multiple index

Elastic Stack Beats
,
1

I use elasticsearch 8.3.3 and filebeat 8.3.3
I try read logs from different docker contaiters by using filebeat and when direct write to different indices to elasticsearch.
My filebeat.yml

# ============================== Filebeat inputs ===============================
filebeat.autodiscover:
  providers:
    - type: docker
      templates:
#STEP2
        - condition:
            contains:
              docker.container.name: img-step2
          config:
            - type: container
              paths:
                - /var/lib/docker/containers/${data.docker.container.id}/*.log
              fields:
                step: STEP2
              fields_under_root: true
#STEP4
        - condition:
            contains:
              docker.container.name: img-step4
          config:
            - type: container
              paths:
                - /var/lib/docker/containers/${data.docker.container.id}/*.log
              fields:
                step: STEP4
              fields_under_root: true
#STEP10
        - condition:
            contains:
              docker.container.name: img-step10
          config:
            - type: container
              paths:
                - /var/lib/docker/containers/${data.docker.container.id}/*.log
              fields:
                step: STEP10
              fields_under_root: true

# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 2
setup.template.name: "images-tst"
setup.template.pattern: "images-tst-*"
#setup.template.overwrite: true

# ================================== Outputs ===================================
output.elasticsearch:
  hosts: ["172.61.10.49"]
  protocol: https
  username: "filebeat"
  password: "tLWeILiKIrBGyVVpBUK1"

  ssl:
    enabled: true
    ca_trusted_fingerprint: "F4EA3BC74EF1BC8CFFF7685FC4B4F577A90A10DBC822A87DCDB956470AA91706"

#  index: "images-tst-%{+yyyy.MM.dd}"

  indices:
    - index: "images-tst-step2-%{+yyyy.MM.dd}"
      when.equals:
        fields:
          step: "STEP2"
    - index: "images-tst-step4-%{+yyyy.MM.dd}"
      when.equals:
        fields:
          step: "STEP4"
    - index: "images-tst-step10-%{+yyyy.MM.dd}"
      when.equals:
        fields:
          step: "STEP10"

#output.console:
#  pretty: true

# ================================= Processors =================================
processors:
  - drop_fields:
      fields: [ "host", "docker"]
      ignore_missing: true

# ================================== Logging ===================================
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0640

But no index is created.

2

Hi @pauldon2,

Filebeat indexes are managed by ILM (index lifecycle management). If you want to use custom indices you probably need to disable it.

3

Thanks a lot. It's works. P.S. Strange but that is not described in any example that I read.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.