ILM is not being disabled in Metricbeat 7.2.1 and 7.3.1

Elastic Stack Beats
1

I am trying to disable ILM in metricbeat 7.2.1 and 7.3.1 in output elasticsearch section but it doesn't work.

metricbeat.config.modules:
    path: ${path.config}/modules.d/*.yml
output:
    elasticsearch:
        hosts:
        - 192.168.33.10:9200
        setup.ilm.enabled: false
        setup.ilm.check_exists: false
        index: metricbeat-%{[metricset.module]}-%{+YYYY}.week.%{+ww}
        indices:
        -   index: metricbeat-%{[metricset.module]}-%{+yyyy.MM.dd}
            when.equals:
                metricset.module: system
        password: pass
        protocol: http
        username: user
setup.dashboards.enabled: true
setup.dashboards.index: metricbeat-*
setup.kibana:
    host: 192.168.33.10:5601
    password: pass
    protocol: http
    username: user
setup.template.fields: /etc/metricbeat/fields.yml
setup.template.name: metricbeat7
setup.template.overwrite: false
setup.template.pattern: metricbeat-*

What is wrong? Is there a bug for that?

2

I can't tell the structure of your config file. Can you please format it as code?

The setup options belong at the root, not under the elasticsearch output section. You can see the required structure in the reference.yml file here: https://www.elastic.co/guide/en/beats/metricbeat/7.3/metricbeat-reference-yml.html

3

@dedemorton

I've formatted my code. Sorry for that.
Yes I tried to disable ILM via setup in root, but in this case I catch ERROR because of which indices are not being sent to Elastic :

2019-10-02T20:07:05.645Z        INFO    [publisher]     pipeline/retry.go:155   Drop batch
2019-10-02T20:07:07.393Z        ERROR   pipeline/output.go:121  Failed to publish events: temporary bulk send failure
4

I think you have a few problems with your config:

  • setup.ilm settings belong at the root.
  • metricset.module doesn't exist in the output as far as I can see. Did you try event.module?
  • possibly some indentation errors (I'm not great at eyeballing stuff).

This indices setting in 7.3.2 worked for me:

      indices:
        - index: "metricbeat-%{[event.module]}-%{+yyyy.MM.dd}"
          when.equals:
            event.module: "system"
5

If you're working from an older config or example, note that there were quite a few naming changes for fields in 7.0 with the migration to the Elastic Common Schema: https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-7.0.html

6

Here's the simplified config that worked for me on 7.3.2 (mostly defaults):

setup.ilm.enabled: false
setup.ilm.check_exists: false
metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.template.settings:
  index.number_of_shards: 1
  index.codec: best_compression
setup.kibana:
cloud.id: "MyStack:somelongstring"
cloud.auth: "elastic:mypass"
output.elasticsearch:
  hosts: ["localhost:9200"]
  indices:
    - index: "metricbeat-%{[event.module]}-%{+yyyy.MM.dd}"
      when.equals:
        event.module: "system"
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
7

@dedemorton,

thank you so much! The problem in my syntax when I tried setup ILM in root was that instead of metricset.module I should use event.module like you mentioned. Now it works as it should. Thanks a lot again!

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.