Filebeat 8.5 pattern not applying to index + the Indices shows as .ds-<name>?

Elastic Stack Beats
,
1

Hello all,

I am using the new Filebeat Helm (8.50 to collect logs from several microservices and send them directly to elasticsearch (EFK).

I've edited the Values.yaml under daemonset with the following:

  filebeatConfig:
    filebeat.yml: |
      filebeat.inputs:
      - type: container
        paths:
          - /var/log/containers/*.log
        processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"

      output.elasticsearch:
        host: '${NODE_NAME}'
        hosts: '["https://${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}"]'
        username: '${ELASTICSEARCH_USERNAME}'
        password: '${ELASTICSEARCH_PASSWORD}'
        protocol: https
        ssl.certificate_authorities: ["/usr/share/filebeat/certs/ca.crt"]
        index: "test"
      setup.template:
        name: "test"
        pattern: "test"
      setup.ilm:
        enabled: true
        rollover_alias: "test"
        pattern: "000001"
        policy_file: /usr/share/filebeat/ilm.json
        overwrite: true
        policy_name: "daily"
      setup.dashboards:
        enabled: true
        index: "test*"
      setup.kibana.host: kibana:5601 

    ilm.json: |
      {
        "policy": {
          "phases": {
            "hot": {
              "actions": {
                "rollover": {
                  "max_age": "1d"
                }
              }
            },
            "delete": {
              "min_age": "7d",
              "actions": {
                "delete": {}
              }
            }
          }
        }
      }

The ILM "daily" is created and Linked to index test, and to Indices ".ds-test-2022.12.01-000001" instead of "test-000001".
Also since it start with .ds it is hidden, which is not convinent.

  • Can you help me understand what am I missing on the setting?
  • Should I have also edited it under the Deployment in the helm.yaml?

Thank you.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.