Hability to create daily index to metricbeat

Hi all,
We are deploying an stack monitoring to eck:

Current monitoring are working fine, to Elasticsearch and kibana resources. But we have a problem with the index creation: metricbeat create a single index with format:

metricbeat-7.16.2-2022.03.09-000001

We want to change this to create a day index with the information about this day, somehting like metricbeat-7.16.2-stack-2022.03.09, metricbeat-7.16.2-stack-2022.03.10, ... to manage it with ILM polycies.

There're some way to change this configuration in this yml file conf to modify the index name creation?

Regards,

Hi @gerardgorrion !

did you try to change output.elasticsearch.index as described here? I think it should be possible to add mentioned configuration parameters under spec.config.

maybe it is also possible to define setup.ilm.* configuration in spec.config instead?

Hi @Tetiana_Kravchenko , we try to deploy with configmap with metricbeat.yml config and no work, the metricbeat no report any information when we use this config:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: metricbeat-config
  namespace: default
  labels:
    k8s-app: metricbeat
data:
  metricbeat.yml: |-
    metricbeat.config.modules:
      # Mounted `metricbeat-daemonset-modules` configmap:
      path: ${path.config}/modules.d/*.yml
      # Reload module configs as they change:
      reload.enabled: false

    processors:
      - add_cloud_metadata: ~
      - add_docker_metadata: ~

    setup.template.name: "metricbeat"
    setup.template.pattern: "metricbeat-*"
    setup.ilm.enabled: false

    output.elasticsearch:
      hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
      username: '${ELASTICSEARCH_USERNAME:}'
      password: '${ELASTICSEARCH_PASSWORD:}'
      index: "metricbeat-%{[agent.version]}-monitoring-%{+yyyy.MM.dd}"

And mapping into beat deploy:

          volumeMounts:
            - name: config
              mountPath: /usr/share/metricbeat/metricbeat.yml
              readOnly: true
              subPath: metricbeat.yml
        volumes:
        - name: config
          configMap:
            name: metricbeat-config

but is no possible to change this file. Also, we try to deploy the metricbeat.yml file into /etc folder and use this arguments to use it:

        args: [
          "-c", "/etc/metricbeat.yml",
          "-e",
        ]

But metricbeat dowsn't work with this config. some idea?

Thanks!

From the Beat CRD it doesn't seem that you should create your configMap and mount it explicitly:

  1. you should define configuration either under spec.config
  2. or under spec.configRef - but it is for Secret k8s resource:
ConfigRef contains a reference to an existing Kubernetes Secret

Thanks! we configure this lines into beat config and work fine now!

    setup.template.name: "metricbeat"
    setup.template.pattern: "metricbeat-*"
    setup.ilm.enabled: false
    output:
      elasticsearch:
        index: "metricbeat-%{[agent.version]}-monitoring-%{+yyyy.MM.dd}"

Now, we only need to monitoring logstash agents, apm-server and resolve the incident with kibana, and we wil finish the migration to metricbeat monitoring.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.