Metricbeat ILM enabled with OSS ES Cluster when using auto

Elastic Stack Beats
1

Hi,

I have an issue with Metricbeat OSS v7.6.1 enabling the ILM feature when the output is an OSS\OD cluster.

Jun 01 10:44:38 ip-10-100-129-179.eu-west-1.compute.internal metricbeat[3654]: 2020-06-01T10:44:38.720Z        ERROR        pipeline/output.go:100        Failed to connect to backoff(elasticsearch(http://developer-data.private.co.uk:9200)): Connection marked as failed because the onConnect callback failed: request checking for ILM availability failed: 500 Internal Server Error: {"error":{"root_cause":[{"type":"security_exception","reason":"Unexpected exception indices:admin/get"}],"type":"security_exception","reason":"Unexpected exception indices:admin/get"},"status":500}

This is easily worked around by setting currently:

setup.ilm.enabled: false

Config:

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml

output.elasticsearch:
  hosts: ["developer-data.private.co.uk:9200"]
  username: "metricbeat_user"
  password: "password"

metricbeat.modules:
  - module: system
    metricsets:
      - cpu
      - filesystem
      - memory
      - network
      - process
      - load
    enabled: true
    period: 10s
    processes: ['.*']
    cpu.metrics: [ percentages, normalized_percentages, ticks ]

Not sure if this is by design due to OpenDistro triggering auth (i guess a proxy auth frontend could also do this if so) but this is triggering a licensed feature in an unlicensed environment. I would have thought that the OSS version of Metricbeat (and any beats for that matter) alone would not allow these features at all never mind the remote being OSS.

I assumed this was a bug as ILM was being enabled before confirming the licence but it appears not.

2

ILM is part of the OSS packages, so users can create their own Beat and to connect to non-OSS Elasticserach instances. However, this behaviour is a bug based on the documentation of auto for ilm:

When auto (the default) is specified on version 7.0 and later, Metricbeat automatically uses index lifecycle management if the feature is enabled in Elasticsearch and has the required license; otherwise, Metricbeat creates daily indices.

Beats should be able to run setup without changing the setting setup.ilm.enabled from auto to false when connecting to an OSS ES. Do you mind reopening the issue and rewording it as a bug report instead of a question?

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.