Security error with beats_system account and Filebeat with system module

Elastic Stack Beats
1

Hey,

I'm encountering an error which indicates that the builtin beats_system user account doesn't have enough permissions in the ES cluster to perform a particular task, though it works when I test with a superuser account. I was expecting the builtin account to have permissions to do what it needs for supported modules in default states.

I've installed FIlebeat 6.4 on CentOS7 and enabled the system module from cli 'sudo filebeat modules enable system'.

I've got x-pack configured and I've configured filebeat to use the beats_system account and output to elasticsearch cluster operating 6.4.

I tested filebeat ingestion into elasticsearch using superuser creds and it worked fine, I also used these to execute filebeat setup as well successfully.

I've setup a keystore and added the variable I'm using for the password.

The settings I've configured in filebeat.yml are:

filebeat.config.modules:
  reload.enabled: true
setup.kibana:
  host: "kibana.mydomain"
output.elasticsearch:
  host: ["es1.mydomain:9200","es2.mydomain:9200"]
  protocol: "http"
  username"beats_system"
  password: "${filebeat_pwd}"
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch:

This is the log error I'm getting:

2018-08-31T17:28:00.967+1000 ERROR pipeline/output.go:91 Failed to connect: Connection marked as failed because the onConnect callback failed: Error loading pipeline for fileset system/auth: couldn't load pipeline: couldn't load json. Error: 403 Forbidden: {"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:admin/ingest/pipeline/put] is unauthorized for user [beats_system]"}],"type":"security_exception","reason":"action [cluster:admin/ingest/pipeline/put] is unauthorized for user [beats_system]"},"status":403}. Response body: {"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:admin/ingest/pipeline/put] is unauthorized for user [beats_system]"}],"type":"security_exception","reason":"action [cluster:admin/ingest/pipeline/put] is unauthorized for user [beats_system]"},"status":403}

2

I've got a fix, but am wondering if this should be part of the beats_system role by default?

I looked through the xpack security privileges for users and found the permission needed. I've created a new account/role and given the role Monitor, Manage_index_templates, Manage_ingest_pipelines permissions. Monitor = same as beats_system role, manage_index_templates to enable it to load index template for beat, manage_ingest_pipelines cause it clearly wanted it in order to load it's pipeline.

3

The documentation states in:
https://www.elastic.co/guide/en/elastic-stack-overview/6.4/built-in-roles.html

beats_system
This role does not provide access to the beats indices and is not suitable for writing beats output to Elasticsearch.

2 Likes
4

I missed that point, that's exactly the clarification I needed, thanks @paltryeffort.

Recap - builtin '*_system' accounts are for xpack endpoint monitoring, not for operational data handling/application functionality.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.