Metricbeat user and Elasticsearch connection

[prodrg]

I have been configuring Metricbeat on my ELK configuration using these two documents:
Securing Communication With Elasticsearch
Beats and Security

The Metricbeat user I configured is based on the Packetbeat user example on Beats and Security, the following way:

POST /_xpack/security/role/metricbeat_writer
{
  "cluster": ["manage_index_templates", "monitor"],
  "indices": [
    {
      "names": [ "metricbeat-*" ], 
      "privileges": ["read","write","create_index"]
    }
  ]
}

POST _xpack/security/role/metricbeat_reader
{
  "indices": [
    {
      "names": [ "metricbeat-*" ], 
      "privileges": ["read","view_index_metadata"]
    }
  ]
}

POST /_xpack/security/user/metricbeat_internal
{
  "password" : "<password>",
  "roles" : ["metricbeat_writer", "metricbeat_reader"],
  "full_name" : "Internal Metricbeat User"
}

When I execute Metricbeat on my system, I receive the following message repeated times:

2017-02-21T14:22:38-06:00 WARN Can not index event (status=403): {"type":"security_exception","reason":"action [indices:admin/create] is unauthorized for user [metricbeat_internal]"}

And even assigning the create privilege, it keeps throwing the same message.
And I guess it is receiving the information properly, because I also receive this:

2017-02-21T14:44:49-06:00 INFO Non-zero metrics in the last 30s: fetches.system-filesystem.success=1 fetches.system-fsstat.events=1 libbeat.es.publish.read_bytes=40005 fetches.system-filesystem.events=13 fetches.system-load.success=1

What should I do?

Thanks in advance

EDIT: I will also add part of what I have configured in metricbeat.yml

#================================ Outputs =====================================

# Configure what outputs to use when sending the data collected by the beat.
# Multiple outputs may be used.

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["https://<my_hostname>"]

  # Optional protocol and basic auth credentials.
  username: "metricbeat_internal"
  password: "<password>"
  ssl.enabled: true
  ssl.verification_mode: none
  ssl.certificate_authorities:
    - <my_path_to_ca.crt>
  template.name: "metricbeat"
  template.path: "<path_to_metricbeat.template.json>"
  template.overwrite: false

[prodrg]

This is mostly a workaround I did. I gave somehow a lot of privileges to the Metricbeat user I created, and it seems it works alright. I got it working on Kibana and the logs are being shown.

So I guess it has to be a bug, because when I assign the create privilege to a role which is assigned to the Metricbeat user, it still sends the same message. I will open an issue on Github.

[ruflin]

Can you try to add create privileges? This should allow to also load the template: https://www.elastic.co/guide/en/x-pack/current/security-privileges.html

[prodrg]

Thanks @ruflin. I did that before, but the same error was given. Although, it seems the issue has been solved, and this is what I did:

After creating the user and adding the superuser role to it, I created a new index pattern on Kibana called metricbeat-*, so it showed all the data received from Metricbeat. Besides, I also updated my stack version from 5.2.0 to 5.2.1.

Now, I stopped metricbeat service, and also deleted the roles metricbeat_writer and metricbeat_reader and the user metricbeat_internal to later recreate them all, but now adding the create privilege to metricbeat_writer. Luckily, the error message isn't showing up. I don't know if it is related to the version upgrade or to the index pattern creation.

[ruflin]

Glad it works. Unfortunately I'm not an export on the security x-pack behaviour. If you are interested in what went wrong and why it is probably best to post it in https://discuss.elastic.co/c/x-pack.

[prodrg]

Hey @ruflin, found again the security exception problem. It is not related to the Beats but totally to X-Pack, so I will close the thread here and marked as solved. Thanks for the support!