# Kibana service keeps failing after upgrade from 7.17 to 8.7

**URL:** https://discuss.elastic.co/t/kibana-service-keeps-failing-after-upgrade-from-7-17-to-8-7/333099
**Category:** Kibana
**Created:** [May 10, 2023, 3:02pm UTC](https://discuss.elastic.co/t/kibana-service-keeps-failing-after-upgrade-from-7-17-to-8-7/333099 "2023-05-10T15:02:01Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![A\_Abdellah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_abdellah/32/100734_2.png) [@A\_Abdellah](https://discuss.elastic.co/u/A_Abdellah)
#### Post date: [May 10, 2023, 3:02pm UTC](https://discuss.elastic.co/t/kibana-service-keeps-failing-after-upgrade-from-7-17-to-8-7/333099/1 "2023-05-10T15:02:01Z")

</div>

Hi,  
I'm trying to upgrade my cluster from 7.17.7 to 8.7 and I'm testing the upgrade on a single node cluster.  
I treated all the errors found by the upgrade assistant and upgraded elasticsearch just fine and tried without upgrading kibana and all works fine.

Kibana assistant API results

```auto
curl -ks -u user:password https://kibana_host:kibana_port/api/upgrade_assistant/status?pretty
{"readyForUpgrade":true,"details":"All deprecation warnings have been resolved."}

```

After I upgraded kibana, the kibana service is stoping repeatedly and when I try to run in using

> sudo -u kibana /usr/share/kibana/bin/kibana

kibana is working fine and connected to elasticsearch and everything.  
The error I'm getting is the following

```auto
{"@timestamp":"2023-05-10T09:57:53.680Z","event":{"provider":"eventLog","action":"stopping"},"message":"eventLog stopping","ecs":{"version":"1.8.0"},"kibana":{"server_uuid":"190aeaf3-8ecc-4ea1-bf95-175946e623f8","version":"8.7.1"}}

[2023-05-10T09:57:53.686+00:00][ERROR][plugins.eventLog] Error: Error writing some bulk events
    at ClusterClientAdapter.indexDocuments (/usr/share/kibana/x-pack/plugins/event_log/server/es/cluster_client_adapter.js:86:23)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at /usr/share/kibana/x-pack/plugins/event_log/server/es/cluster_client_adapter.js:44:216
[
  {
    create: {
      _index: '.kibana-event-log-8.7.1',
      _id: 'bjAYBYgBkRNw1UIoh9aT',
      status: 403,
      error: {
        type: 'cluster_block_exception',
        reason: 'index [.kibana-event-log-8.7.1-000001] blocked by: [FORBIDDEN/8/index write (api)];'
      }
    }
  }
]

```

I would like to note that I enabled and configured the ssl for all elements before the upgrade so It's not using elastic 8 auto-configured security.

Do you have any recommandations for my situation ?  
Also If you have found any challenges with your upgrades from 7.17 to 8 please share them with me.

---

<div class="post-metadata">

### Author: ![A\_Abdellah](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/a_abdellah/32/100734_2.png) [@A\_Abdellah](https://discuss.elastic.co/u/A_Abdellah)
#### Post date: [May 12, 2023, 10:00pm UTC](https://discuss.elastic.co/t/kibana-service-keeps-failing-after-upgrade-from-7-17-to-8-7/333099/2 "2023-05-12T22:00:18Z")

</div>

Hello everyone,  
Alhamdulillah,  
I found the solution for this issue, after installing kibana 8.7.1 on another host and comparing I foundout that the `/etc/systemd/system/kibana.service` file has been updated in the newer version but installing the upgrade using apt did not handel this. please find below the old and new service definitions.

> Kibana 7.17.7 service

```auto
[Unit]
Description=Kibana

[Service]
Type=simple
User=kibana
Group=kibana
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
EnvironmentFile=-/etc/default/kibana
ExecStart=/usr/share/kibana/bin/kibana "-c /etc/kibana/kibana.yml"
Restart=always
WorkingDirectory=/

[Install]
WantedBy=multi-user.target

```

> Kibana 8.7.1 service

```auto
# /lib/systemd/system/kibana.service
[Unit]
Description=Kibana
Documentation=https://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=kibana
Group=kibana
PrivateTmp=true

Environment=KBN_HOME=/usr/share/kibana
Environment=KBN_PATH_CONF=/etc/kibana

EnvironmentFile=-/etc/default/kibana
EnvironmentFile=-/etc/sysconfig/kibana

ExecStart=/usr/share/kibana/bin/kibana

Restart=on-failure
RestartSec=3

StartLimitBurst=3
StartLimitInterval=60

WorkingDirectory=/usr/share/kibana

StandardOutput=journal
StandardError=inherit

[Install]
WantedBy=multi-user.target

```

After updating the file you will need to run the command

```auto
systemctl daemon-reload

```

for the `kibana.yml` the only change needed is the new logging configuration as mentioned on the upgrade documentation.

```auto
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file

```

I hope this would be helpfull.  
Thank you.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 16, 2023, 12:17am UTC](https://discuss.elastic.co/t/kibana-service-keeps-failing-after-upgrade-from-7-17-to-8-7/333099/3 "2023-05-16T00:17:04Z")

</div>

Thanks for sharing the solution! 😃

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 13, 2023, 12:17am UTC](https://discuss.elastic.co/t/kibana-service-keeps-failing-after-upgrade-from-7-17-to-8-7/333099/4 "2023-06-13T00:17:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
