Metricbeat ILM issue

[ManuelF (Manuel)]

Hi,

*Note: Running ELK 7.9.2. Beats version matches ELK version.

I recently deployed Metricbeat to monitor my MSSQL server. Setup was successfully completed according to the PS console output. Metricbeat index, index template and ILM policy where successfully added to ELK. Other beats were also successfully deployed following exactly the same procedure. All other beat indexes, except Metricbeat where assigned an ILM policy. If I check the index template settings, system states that Metricbeat ILM is already assigned to Metricbeat index template, however the index does not show any ILM assigned to it, while index template does. ILM was automatically assigned to other beats, but for some reason it was different for Metricbeat. Please see screenshots below.

Is this a normal behavior or, perhaps am I missing something?

Thank you

[ManuelF (Manuel)]

Update

• Stopped Metricbeat service in MSSQL server
• Removed Metricbeat index from ES
• Restarted Logstash
• After confirmed that metricbeat-7.9.2 index template has its own ILM assigned in the template config, I restarted the Metricbeat service in the MSSQL server.

Checking the beat logs I noticed that connection for monitoring was successfully established, but the beat connection for the "real" data is now failing due to a permissions issue:

"security_exception","reason":"action [indices:admin/aliases] is unauthorized for API key id [Tzi6iXUB5EDWlYDvCLAd] of user [elastic]"},"status":403}: 403 Forbidden"

This is happening only with Metricbeat. I am using API permissions to access ES and used the same permissions config for each API, but just changed the name for each beat, so each beat has it's own API credentials, but with the same cluster permissions and same index permissions to manage their own index.

Why only Metricbeat is experiencing access permission issues?
Does Metricbeat need additional permissions?

These are the permissions set for the API (same for the other beats):

POST /_security/api_key
{
  "name": "metricbeat-W", 
  "role_descriptors": {
    "beats_writer": { 
      "cluster": ["monitor", "read_ilm", "manage_ingest_pipelines"],
      "index": [
        {
          "names": ["metricbeat-*"],
          "privileges": ["create_index", "create_doc", "view_index_metadata"]
        }
      ]
    }
  }
}

[ManuelF (Manuel)]

Any comment or advise on this issue?
Thanks