Windows filebeat cluster privileges

[mcosta]

Hi,

I installed Filebeat (7.16.2) on Windows Server, using default PowerShell script and started it as a service.

It reads some custom log files and send them to Elasticsearch. It uses a custom index, ILM policy and template so I had some initial problems dealing with Filebeat configuration but now it works fine. It also uses a newly created user having his password stored on a keystore to avoid plain text passwords on configuration file.

As far as I could test it only runs (start and update index when there are new records on log files) if user's role have the following set of privileges:

• Cluster: read_ilm, monitor, manage_index_templates
• Index: create_index, view_index_metadata, write

All this was a test to find both Filebeat and Elastcisearch configurations. QA and Production deployments of this solution will be done by someone else. I don't even have access to the Windows server where Filebeat will run. So, my main questions are:

Is it possible to run with less privileges? Am I doing anything wrong that should be done some other way?
For instance, "manage_index_templates" will allow this user to manage all other index templates on this Cluster, right? That could be a security problem.

Thank you and HAPPY NEW YEAR

[kvch (Noémi Ványi)]

Have you seen this documentation page about giving access to resources on ES and why specific roles are needed? Grant users access to secured resources | Filebeat Reference [7.16] | Elastic

Let me know if you have further questions.

[mcosta]

Hi Noémi

Thank you for your reply.

Somehow I missed that documentation but thanks to the great explicit error logging output from Filebeat, I was able to tune the above privileges.

Refering to documentation, suggested "writer role" needed for publishing, includes cluster monitor, read_ilm and read_pipeline privileges but not manage_index_templates. Without manage_index_templates set for this user's role I get the error:

ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(Elasticsearch(https://elasticq.bdso.local:9200)): Connection marked as failed because the onConnect callback failed: error loading template: failure while checking if template exists: 403 Forbidden

Any suggestion?

Thanks!