# Permissions issues after upgrading heartbeat to 8.0

**URL:** https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383
**Category:** Beats
**Tags:** heartbeat
**Created:** [February 16, 2022, 3:20pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383 "2022-02-16T15:20:36Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Marc-Antoine\_J](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marc-antoine_j/32/39687_2.png) [@Marc-Antoine\_J](https://discuss.elastic.co/u/Marc-Antoine_J)
#### Post date: [February 16, 2022, 3:20pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/1 "2022-02-16T15:20:36Z")

</div>

Good Day,

I recently upgrade my test installation of the Elastic Stack to 8.0. The Elasticsearch and kibana installations went fine. On the heartbeat side I'm getting the following errors:

```auto
Cannot index event publisher.Event {removed data} \"reason\":\"action [indices:admin/mapping/auto_put] is unauthorized for user [heartbeat_writer] with roles [heartbeat_writer] on indices [.ds-heartbeat-8.0.0-2022.02.16-000001], this action is granted by the index privileges [auto_configure,manage,write,all]\"}, dropping event!","service.name":"heartbeat","ecs.version":"1.6.0"

```

I updated through RPM package since my system is offline (no internet access). Just in case I've re-run the heartbeat setup with my yml file with a user (elastic) which has all privileges. I've set my heartbeat-writer role with the permissions explained in the heartbeat docs:  
[https://www.elastic.co/guide/en/beats/heartbeat/current/privileges-to-publish-events.html](https://www.elastic.co/guide/en/beats/heartbeat/current/privileges-to-publish-events.html)

My yml is very simple. I kept defaults but for the Elasticsearch output for the server ip, user/password. All my monitors are configured in the monitors.d folder and should not have any impact on this topic.

There's no index "heartbeat-8.0.0" that has created but a ".ds-heartbeat-8.0.0" appeared with just a few documents. 53 for now (for the last 10 hours) and I used to have 1656 docs per day before the upgrade. The current docs are all timeouts probably when I was doing firewall reloads.

Did I miss that now the index naming is .ds-beat-version etc?

Thanks,

---

<div class="post-metadata">

### Author: ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [February 23, 2022, 9:41pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/2 "2022-02-23T21:41:01Z")

</div>

Apologies for the poor experience @Marc-Antoine_J , and another apology for the delay in responding here.

We can do better here, this is a result of this [breaking change in 8.0](https://www.elastic.co/guide/en/beats/libbeat/current/breaking-changes-8.0.html#_beats_now_store_events_in_data_streams_instead_of_indices), where we now store events in [data streams](https://www.elastic.co/guide/en/elasticsearch/reference/8.0/data-streams.html) by default.

Data streams really are superior, but you'll need to modify your permissions to match the the error message shown. You can do that by adding the `manage_index_templates` cluster permission to the `hearbeat_writer` role.

I've opened [beats#30567](https://github.com/elastic/beats/pull/30567) this PR which fixes the docs as well.

---

<div class="post-metadata">

### Author: ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [February 23, 2022, 9:41pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/3 "2022-02-23T21:41:58Z")

</div>

I should mention that you don't really need to change the index level perms to account for the `.ds-*` prefix, that's an internal detail that's hidden behind the datastream abstraction in most cases, but leaks through in some places.

---

<div class="post-metadata">

### Author: ![Marc-Antoine\_J](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marc-antoine_j/32/39687_2.png) [@Marc-Antoine\_J](https://discuss.elastic.co/u/Marc-Antoine_J)
#### Post date: [February 24, 2022, 2:21pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/4 "2022-02-24T14:21:32Z")

</div>

Hi @Andrew_Cholakian1 , thanks for trying to help. It seems that I had tried to add "manage\_index\_templates" to my role. This morning I've tried to restart my heartbeat service just to make sure the permissions are applied correctly but I still experience the exact same issues.

So if I understand correctly my role still has to apply permissions on heartbeat-\* indexes and does not need to apply any on .ds-\* since this is all in the datastream abstraction.

Any clues about why I still have the issue?

Thanks!

---

<div class="post-metadata">

### Author: ![Marc-Antoine\_J](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marc-antoine_j/32/39687_2.png) [@Marc-Antoine\_J](https://discuss.elastic.co/u/Marc-Antoine_J)
#### Post date: [February 24, 2022, 2:31pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/5 "2022-02-24T14:31:06Z")

</div>

Here's a JSON formatted journalctl log with maybe more information. Hope this helps:

```auto
Feb 24 09: 44: 52 --Server Name -- heartbeat[465243]: {
    "log.level": "warn",
    "@timestamp": "2022-02-24T09:44:52.256-0500",
    "log.logger": "elasticsearch",
    "log.origin": {
        "file.name": "elasticsearch/client.go",
        "file.line": 414
    },
    "message": "Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Date(2022, time.February, 24, 9, 44, 51, 132645113, time.Local), Meta:null, Fields:{--Removed Data--}}, Private:interface {}(nil), TimeSeries:false}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(nil)}} (status=403): {\"type\":\"security_exception\",\"reason\":\"action [indices:admin/mapping/auto_put] is unauthorized for user [heartbeat_writer] with roles [heartbeat_writer] on indices [.ds-heartbeat-8.0.0-2022.02.16-000001], this action is granted by the index privileges [auto_configure,manage,write,all]\"}, dropping event!",
    "service.name": "heartbeat",
    "ecs.version": "1.6.0"
}

```

---

<div class="post-metadata">

### Author: ![Andrew\_Cholakian1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrew_cholakian1/32/3612_2.png) [@Andrew\_Cholakian1](https://discuss.elastic.co/u/Andrew_Cholakian1)
#### Post date: [February 24, 2022, 11:55pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/6 "2022-02-24T23:55:15Z")

</div>

That's really surprising, I tried replicating the issue myself yesterday and found that change did fix things. I got the error without that perm, and didn't when I added it.

Could you try adding those perms to `.ds-heartbeat-*` and let us know if that helps. It won't hurt anything, and it'd be interesting to see if that fixes it.

---

<div class="post-metadata">

### Author: ![Marc-Antoine\_J](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marc-antoine_j/32/39687_2.png) [@Marc-Antoine\_J](https://discuss.elastic.co/u/Marc-Antoine_J)
#### Post date: [February 25, 2022, 1:16am UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/7 "2022-02-25T01:16:20Z")

</div>

@Andrew_Cholakian1 thanks for taking the time to troubleshoot our issue. It did not change anything to add .ds-heartbeat-\* to the role. I did not expect either since it seems a cluster permission issue more than an index permissions issue. Maybe I'm missing something on my role setup. from the log you can see the role used. I'm attaching a screenshot of the kibana view of the role setup.

 ![heartbeat_writer_permissions](https://us1.discourse-cdn.com/elastic/original/3X/5/7/57ebe056808b8a103988e968740283257d8eb090.png)

Thanks,

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 25, 2022, 3:16am UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/8 "2022-03-25T03:16:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [March 29, 2022, 1:48pm UTC](https://discuss.elastic.co/t/permissions-issues-after-upgrading-heartbeat-to-8-0/297383/9 "2022-03-29T13:48:46Z")

</div>

I think this fix will solve your issue: [Put data stream so there is no need for additional permissions by kvch · Pull Request #31048 · elastic/beats · GitHub](https://github.com/elastic/beats/pull/31048)
