Beats privileges documentation

[jsu]

Hi there,

I had a problem that I managed to resolve but it seems then that the documentation isn't completely accurate. Let me know if I should post this somewhere else.

So, when configuring Beats agents to send data to a cluster with security activated, I followed this documentation and configured a user having a "writer role" with the privileges create_doc and view_index_metadata for all the metricbeat-*, filebeat-* and other *beat-* as specified in the doc. The Beats were able to connect to the cluster but no data was coming in.

The solution was to actually give this role the create_doc and view_index_metadata privileges for all indices * and the data began flowing. As far as I know, it's not mentioned anywhere in the doc that privileges need to be applied to more than the specific *beat-* indices.

Is it indeed a problem with the documentation ?

ps : my indices do have the regular patterns *beat-*.
pps : I linked the doc for Metricbeat, but it's the same for others (here for Auditbeat)

[dedemorton]

Thanks for reporting this issue. I will see if I can reproduce the problem. view_index_metadata on all indices was not required when I tested the privileges several months ago, but it's possible that something changed under the hood, or maybe the content changed.

Note that there's also a github issue open that indicates problems with the documented privileges: https://github.com/elastic/beats/issues/15040.

I'll update the github issue to remind me to post back on this thread when I've figured out the problem. Thanks again!

[jsu]

Thanks for your answer !

Actually the problem should mainly be the create_doc on all indices if ILM is enabled.

Thanks for the github issue, couldn't find it while I checked before posting here.

Would you prefer me to post there ? And should I post github issues instead of here when I encounter things like this ?

[dedemorton]

It's a good idea to post to this forum first in case the problem is user error.

Thanks again for your input. I'll take a look at this as soon as I'm able.

[dedemorton]

Hi. Just an update that I was unable to reproduce this problem. However I did notice that the docs were ambiguous (it wasn't clear that the privileges documented for the writer role assume that you've already run the setup using the roles described for setup).

See https://github.com/elastic/beats/pull/15604.

[jsu]

Thanks a lot for your answers.

After reading that you weren't able to reproduce the problem, I went back to testing and I actually found out what was wrong.

The documentation mentions to give the create_doc privilege to all metricbeat-* (I'll take metricbeat as example) indices, which I did as my indices are named following the pattern metricbeat-000001. But, as I have ILM enabled, I created a metricbeat alias that points to the last metricbeat-* index to write, and to all metricbeat-* indices to read.

When I configure the metricbeat_writer role with the create_doc privilege on metricbeat, and not metricbeat-*, everything works !

I understand why giving the privilege on metricbeat works, but I don't understand why giving it to metricbeat-* doesn't. At the end, data is written on an index, not the alias itself.

[jsu]

So, maybe it wasn't really clear with my previous answer, but I was wondering why this behavior is happening.

Do you have any idea @dedemorton ?

[dedemorton]

Sorry about my delayed response. I was on vacation. You can read about alias behavior in the Elasticsearch docs.

The gist is that when Metricbeat sends events to ES, it's using the alias in the API call, not the concrete index behind the alias.

[jsu]

No problem and thanks a lot for the answer ! (and the link that, for some reason, I missed)