# Heartbeat ask for unnecessary cluster privileges at startup

**URL:** <https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814>\
**Category:** Beats\
**Tags:** elastic-stack-security, ilm-index-lifecycle-management, heartbeat\
**Created:** [June 14, 2021, 9:24am UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814 "2021-06-14T09:24:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![iorfix](https://avatars.discourse-cdn.com/v4/letter/i/9fc348/32.png) [@iorfix](https://discuss.elastic.co/u/iorfix)\
**Post date:** [June 14, 2021, 9:24am UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814/1 "2021-06-14T09:24:54Z")

</div>

Hi all,  
I'm using Heartbeat 7.13.1.  
I first setup templates and ILM policy with elastic superuser, and then I start heartbeat using a ristrected "apikey" user. However, I have this error at startup:

```auto
 {"log.level":"error","@timestamp":"2021-06-14T11:06:35.785+0200","log.logger":"index-management.ilm","log.origin":{"file.name":"ilm/std.go","file.line":166},"message":"ILM policy heartbeat creation failed: 403 Forbidden: {\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"action [cluster:admin/ilm/put] is unauthorized for API key id [0e0c9nkBdKUEp4Eqeb4y] of user [apikeywriteruser], this action is granted by the cluster privileges [manage_ilm,manage,all]\"}],\"type\":\"security_exception\",\"reason\":\"action [cluster:admin/ilm/put] is unauthorized for API key id [0e0c9nkBdKUEp4Eqeb4y] of user [apikeywriteruser], this action is granted by the cluster privileges [manage_ilm,manage,all]\"},\"status\":403}","ecs.version":"1.6.0"}

```

Note that I've the following configuration on heartbeat.yml:

```auto
setup.template.enabled: false
setup.ilm.check_exists: false
setup.ilm.overwrite: false
setup.ilm.enabled: true

```

and the policy is already loaded, since the setup command completed successfully

```auto
heartbeat setup -E setup.ilm.overwrite=true -E setup.template.enabled=true -E output.elasticsearch.username=elastic -E output.elasticsearch.password=***

```

I would avoid to give cluster permission to beat users, is there a better solution?  
Note that I use the same pattern (and even the same key) with metricbeat modules (7.10.2) and they worked correctly.  
Thank you

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [June 14, 2021, 1:26pm UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814/2 "2021-06-14T13:26:42Z")

</div>

Have you tried this with metricbeat 7.13.1? Under the hood they use the same code for settings / ES connection

---

<div class="post-metadata">

**Author:** ![iorfix](https://avatars.discourse-cdn.com/v4/letter/i/9fc348/32.png) [@iorfix](https://discuss.elastic.co/u/iorfix)\
**Post date:** [June 14, 2021, 2:01pm UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814/3 "2021-06-14T14:01:38Z")

</div>

Hi @Andrew_Cholakian1 ,  
I confirm I have the same problem with metricbeat 7.13.1, while I haven't with (at least) 7.10.2, but probably also newer version

```auto
2021-06-14T13:58:06.788Z INFO [index-management] idxmgmt/std.go:261 Auto ILM enable success.
2021-06-14T13:58:06.790Z ERROR [index-management.ilm] ilm/std.go:166 ILM policy metricbeat creation failed: 403 Forbidden: {"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:admin/ilm/put] is unauthorized for user [apikeywriteruser] with roles [role_beats_cloud,role_apm_api_key,role_restapiplus_api_key], this action is granted by the cluster privileges [manage_ilm,manage,all]"}],"type":"security_exception","reason":"action [cluster:admin/ilm/put] is unauthorized for user [apikeywriteruser] with roles [role_beats_cloud,role_apm_api_key,role_restapiplus_api_key], this action is granted by the cluster privileges [manage_ilm,manage,all]"},"status":403}

```

---

<div class="post-metadata">

**Author:** ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)\
**Post date:** [June 15, 2021, 1:21am UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814/4 "2021-06-15T01:21:20Z")

</div>

Thanks a ton for helping us out there, I'll ping the relevant people, since I'm not super up to date on changes there.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [June 15, 2021, 8:55am UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814/5 "2021-06-15T08:55:49Z")

</div>

This is definitely a bug. With these settings the Beat should not need to talk with Elasticsearch. The error message suggests that the Beat is trying to overwrite the ILM policy. Can you file it on [github](https://github.com/elastic/beats/issues/new?assignees=&labels=&template=bug-report.md) please?

---

<div class="post-metadata">

**Author:** ![iorfix](https://avatars.discourse-cdn.com/v4/letter/i/9fc348/32.png) [@iorfix](https://discuss.elastic.co/u/iorfix)\
**Post date:** [June 15, 2021, 2:28pm UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814/6 "2021-06-15T14:28:43Z")

</div>

Issue opened at:

> <https://github.com/elastic/beats/issues/26322>
>
> Using version 7.13.1 Beats force to set ILM policies, even when configuration ex…plicitly set them to don't set them.
> This may cause an error, if beats credentials are reduced to minimum, as in: https://www.elastic.co/guide/en/beats/filebeat/current/privileges-to-publish-events.html
>  
> \- Version: 7.13.1 (it doesn't affect 7.10.2)
> \- Operating System: CentOs / Redhat
> \- Discuss Forum URL: https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814
> \- Steps to Reproduce: 
> 
> 1. create a user with this privileges, and use it in a metricbeat.yml configuration:
> \`\`\`
> {
> "cluster": \["monitor" ,"cluster:admin/ingest/pipeline/get"\],
> "indices": \[
> {
> "names": \["cloud-metrics\*", "cloud-logs\*", "metricbeat-\*", "filebeat-\*", "heartbeat-\*" \],
> "privileges": \["create\_doc"\]
> }
> \],
> "metadata" : { 
> "version" : 1
> }
> }
> \`\`\`
> 
> 2. execute \`metricbeat setup\` with proper superuser credentials.
> \`\`\`
> metricbeat setup -E setup.ilm.overwrite=true -E setup.template.enabled=true -E output.elasticsearch.username=elastic -E output.elasticsearch.password=\*\*\*
> \`\`\`
> 
> 3. configure metricbeat.yml with these settings:
> 
> \`\`\`
> setup.template.enabled: false
> setup.ilm.check\_exists: false
> setup.ilm.overwrite: false
> setup.ilm.enabled: true
> \`\`\`
>  
> 4. start metricbeat and check for logs. It should appear something similar to:
> 
> \`\`\`
> {"log.level":"error","@timestamp":"2021-06-14T11:06:35.785+0200","log.logger":"index-management.ilm","log.origin":{"file.name":"ilm/std.go","file.line":166},"message":"ILM policy heartbeat creation failed: 403 Forbidden: {\\"error\\":{\\"root\_cause\\":\[{\\"type\\":\\"security\_exception\\",\\"reason\\":\\"action \[cluster:admin/ilm/put\] is unauthorized for API key id \[0e0c9nkBdKUEp4Eqeb4y\] of user \[apikeywriteruser\], this action is granted by the cluster privileges \[manage\_ilm,manage,all\]\\"}\],\\"type\\":\\"security\_exception\\",\\"reason\\":\\"action \[cluster:admin/ilm/put\] is unauthorized for API key id \[0e0c9nkBdKUEp4Eqeb4y\] of user \[apikeywriteruser\], this action is granted by the cluster privileges \[manage\_ilm,manage,all\]\\"},\\"status\\":403}","ecs.version":"1.6.0"}
> \`\`\`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 9, 2021, 2:29pm UTC](https://discuss.elastic.co/t/heartbeat-ask-for-unnecessary-cluster-privileges-at-startup/275814/7 "2021-07-09T14:29:11Z")

</div>

This topic was automatically closed 24 days after the last reply. New replies are no longer allowed.
