I had a problem that I managed to resolve but it seems then that the documentation isn't completely accurate. Let me know if I should post this somewhere else.
So, when configuring Beats agents to send data to a cluster with security activated, I followed this documentation and configured a user having a "writer role" with the privileges create_doc and view_index_metadata for all the metricbeat-*, filebeat-* and other *beat-* as specified in the doc. The Beats were able to connect to the cluster but no data was coming in.
The solution was to actually give this role the create_doc and view_index_metadata privileges for all indices * and the data began flowing. As far as I know, it's not mentioned anywhere in the doc that privileges need to be applied to more than the specific *beat-* indices.
Is it indeed a problem with the documentation ?
ps : my indices do have the regular patterns *beat-*.
pps : I linked the doc for Metricbeat, but it's the same for others (here for Auditbeat)
Thanks for reporting this issue. I will see if I can reproduce the problem. view_index_metadata on all indices was not required when I tested the privileges several months ago, but it's possible that something changed under the hood, or maybe the content changed.
Hi. Just an update that I was unable to reproduce this problem. However I did notice that the docs were ambiguous (it wasn't clear that the privileges documented for the writer role assume that you've already run the setup using the roles described for setup).
After reading that you weren't able to reproduce the problem, I went back to testing and I actually found out what was wrong.
The documentation mentions to give the create_doc privilege to all metricbeat-* (I'll take metricbeat as example) indices, which I did as my indices are named following the pattern metricbeat-000001. But, as I have ILM enabled, I created a metricbeat alias that points to the last metricbeat-* index to write, and to all metricbeat-* indices to read.
When I configure the metricbeat_writer role with the create_doc privilege on metricbeat, and not metricbeat-*, everything works !
I understand why giving the privilege on metricbeat works, but I don't understand why giving it to metricbeat-* doesn't. At the end, data is written on an index, not the alias itself.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.