Hability to create daily index to metricbeat

Elastic Stack Beats
,
1

Hi all,
We are deploying an stack monitoring to eck:

Current monitoring are working fine, to Elasticsearch and kibana resources. But we have a problem with the index creation: metricbeat create a single index with format:

metricbeat-7.16.2-2022.03.09-000001

We want to change this to create a day index with the information about this day, somehting like metricbeat-7.16.2-stack-2022.03.09, metricbeat-7.16.2-stack-2022.03.10, ... to manage it with ILM polycies.

There're some way to change this configuration in this yml file conf to modify the index name creation?

Regards,

2

Hi @gerardgorrion !

did you try to change output.elasticsearch.index as described here? I think it should be possible to add mentioned configuration parameters under spec.config.

maybe it is also possible to define setup.ilm.* configuration in spec.config instead?

3

Hi @Tetiana_Kravchenko , we try to deploy with configmap with metricbeat.yml config and no work, the metricbeat no report any information when we use this config:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: metricbeat-config
  namespace: default
  labels:
    k8s-app: metricbeat
data:
  metricbeat.yml: |-
    metricbeat.config.modules:
      # Mounted `metricbeat-daemonset-modules` configmap:
      path: ${path.config}/modules.d/*.yml
      # Reload module configs as they change:
      reload.enabled: false

    processors:
      - add_cloud_metadata: ~
      - add_docker_metadata: ~

    setup.template.name: "metricbeat"
    setup.template.pattern: "metricbeat-*"
    setup.ilm.enabled: false

    output.elasticsearch:
      hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
      username: '${ELASTICSEARCH_USERNAME:}'
      password: '${ELASTICSEARCH_PASSWORD:}'
      index: "metricbeat-%{[agent.version]}-monitoring-%{+yyyy.MM.dd}"

And mapping into beat deploy:

          volumeMounts:
            - name: config
              mountPath: /usr/share/metricbeat/metricbeat.yml
              readOnly: true
              subPath: metricbeat.yml
        volumes:
        - name: config
          configMap:
            name: metricbeat-config

but is no possible to change this file. Also, we try to deploy the metricbeat.yml file into /etc folder and use this arguments to use it:

        args: [
          "-c", "/etc/metricbeat.yml",
          "-e",
        ]

But metricbeat dowsn't work with this config. some idea?

Thanks!

4

From the Beat CRD it doesn't seem that you should create your configMap and mount it explicitly:

  1. you should define configuration either under spec.config
  2. or under spec.configRef - but it is for Secret k8s resource:
ConfigRef contains a reference to an existing Kubernetes Secret
5

Thanks! we configure this lines into beat config and work fine now!

    setup.template.name: "metricbeat"
    setup.template.pattern: "metricbeat-*"
    setup.ilm.enabled: false
    output:
      elasticsearch:
        index: "metricbeat-%{[agent.version]}-monitoring-%{+yyyy.MM.dd}"

Now, we only need to monitoring logstash agents, apm-server and resolve the incident with kibana, and we wil finish the migration to metricbeat monitoring.