# \[SOLVED\] Kibana crashes on reindex after 6.5.0 upgrade **URL:** \ **Category:** Kibana\ **Created:** [November 16, 2018, 12:16pm UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050 "2018-11-16T12:16:21Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![kribor1](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@kribor1](https://discuss.elastic.co/u/kribor1)\ **Post date:** [November 16, 2018, 12:16pm UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050/1 "2018-11-16T12:16:21Z")
Was on 6.4, upgraded elastic 6.5 & restarted nodes - \> upgrade complete. Upgraded kibana 6.4 -\> 6.5.0 and restarted-\> crash an burn I retried several times after deleting .kibana\_1 and .kibana\_2 ``` log [12:06:46.987] [info][status][plugin:beats_management@6.5.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [12:06:47.024] [info][status][plugin:apm@6.5.0] Status changed from uninitialized to green - Ready log [12:06:47.200] [info][status][plugin:canvas@6.5.0] Status changed from uninitialized to green - Ready log [12:06:47.207] [info][status][plugin:console@6.5.0] Status changed from uninitialized to green - Ready log [12:06:47.209] [info][status][plugin:console_extensions@6.5.0] Status changed from uninitialized to green - Ready log [12:06:47.213] [info][status][plugin:notifications@6.5.0] Status changed from uninitialized to green - Ready log [12:06:47.253] [info][status][plugin:infra@6.5.0] Status changed from uninitialized to green - Ready log [12:06:47.262] [info][status][plugin:metrics@6.5.0] Status changed from uninitialized to green - Ready log [12:06:47.270] [info][status][plugin:elasticsearch@6.5.0] Status changed from yellow to green - Ready log [12:06:47.619] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml log [12:06:47.621] [info][status][plugin:reporting@6.5.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [12:06:47.669] [info][license][xpack] Imported license information from Elasticsearch for the [data] cluster: mode: basic | status: active log [12:06:47.674] [info][status][plugin:xpack_main@6.5.0] Status changed from yellow to green - Ready log [12:06:47.675] [info][status][plugin:searchprofiler@6.5.0] Status changed from yellow to green - Ready log [12:06:47.676] [info][status][plugin:ml@6.5.0] Status changed from yellow to green - Ready log [12:06:47.677] [info][status][plugin:tilemap@6.5.0] Status changed from yellow to green - Ready log [12:06:47.677] [info][status][plugin:watcher@6.5.0] Status changed from yellow to green - Ready log [12:06:47.678] [info][status][plugin:index_management@6.5.0] Status changed from yellow to green - Ready log [12:06:47.678] [info][status][plugin:rollup@6.5.0] Status changed from yellow to green - Ready log [12:06:47.679] [info][status][plugin:graph@6.5.0] Status changed from yellow to green - Ready log [12:06:47.681] [info][status][plugin:grokdebugger@6.5.0] Status changed from yellow to green - Ready log [12:06:47.682] [info][status][plugin:logstash@6.5.0] Status changed from yellow to green - Ready log [12:06:47.682] [info][status][plugin:beats_management@6.5.0] Status changed from yellow to green - Ready log [12:06:47.683] [info][status][plugin:reporting@6.5.0] Status changed from yellow to green - Ready log [12:06:47.683] [info][kibana-monitoring][monitoring-ui] Starting monitoring stats collection log [12:06:47.697] [info][status][plugin:security@6.5.0] Status changed from yellow to green - Ready log [12:06:47.761] [info][license][xpack] Imported license information from Elasticsearch for the [monitoring] cluster: mode: basic | status: active log [12:06:48.374] [info][migrations] Creating index .kibana_2. log [12:06:48.809] [info][migrations] Reindexing .kibana to .kibana_1 error [12:06:49.523] [fatal][root] [resource_not_found_exception] task [k3s35GjtR7ySZkTMIiumgQ:776425] isn't running and hasn't stored its results :: {"path":"/_tasks/k3s35GjtR7ySZkTMIiumgQ%3A776425","query":{},"statusCode":404,"response":"{\"error\":{\"root_cause\":[{\"type\":\"resource_not_found_exception\",\"reason\":\"task [k3s35GjtR7ySZkTMIiumgQ:776425] isn't running and hasn't stored its results\"}],\"type\":\"resource_not_found_exception\",\"reason\":\"task [k3s35GjtR7ySZkTMIiumgQ:776425] isn't running and hasn't stored its results\"},\"status\":404}"} at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:308:15) at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:267:7) at HttpConnector. (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:165:7) at IncomingMessage.wrapper (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4949:19) at emitNone (events.js:111:20) at IncomingMessage.emit (events.js:208:7) at endReadableNT (_stream_readable.js:1064:12) at _combinedTickCallback (internal/process/next_tick.js:138:11) at process._tickCallback (internal/process/next_tick.js:180:9) FATAL [resource_not_found_exception] task [k3s35GjtR7ySZkTMIiumgQ:776425] isn't running and hasn't stored its results :: {"path":"/_tasks/k3s35GjtR7ySZkTMIiumgQ%3A776425","query":{},"statusCode":404,"response":"{\"error\":{\"root_cause\":[{\"type\":\"resource_not_found_exception\",\"reason\":\"task [k3s35GjtR7ySZkTMIiumgQ:776425] isn't running and hasn't stored its results\"}],\"type\":\"resource_not_found_exception\",\"reason\":\"task [k3s35GjtR7ySZkTMIiumgQ:776425] isn't running and hasn't stored its results\"},\"status\":404}"} ``` Ideas? I tried manually reindexing the .kibana to a custom name which works fine: > curl -X POST [http://10.10.20.201:9200/\_reindex](http://10.10.20.201:9200/_reindex) -H 'Content-Type: application/json' -d @- \<\< EOF > { > "source": { > "index": ".kibana" > }, > "dest": { > "index": ".kibana\_kristofer" > } > } > EOF ---
**Author:** ![rashmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rashmi/32/16391_2.png) [@rashmi](https://discuss.elastic.co/u/rashmi)\ **Post date:** [November 16, 2018, 3:15pm UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050/2 "2018-11-16T15:15:41Z")
There is a breaking change after an upgrade to 6.5, have you checked the release notes? ([https://www.elastic.co/guide/en/kibana/current/release-notes-6.5.0.html#known-issues-6.5.0](https://www.elastic.co/guide/en/kibana/current/release-notes-6.5.0.html#known-issues-6.5.0)). Are you facing the same issues? Thanks Rashmi ---
**Author:** ![kribor1](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@kribor1](https://discuss.elastic.co/u/kribor1)\ **Post date:** [November 16, 2018, 4:14pm UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050/3 "2018-11-16T16:14:06Z")
Ah, I did give that a quick scan but it didn't seem applicable since security is a paid and I'm not paying. It does seem related but since I don't have security enabled there are plenty of steps I can't perform... 😮 ---
**Author:** ![tylersmalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tylersmalley/32/8833_2.png) [@tylersmalley](https://discuss.elastic.co/u/tylersmalley)\ **Post date:** [November 16, 2018, 5:53pm UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050/4 "2018-11-16T17:53:31Z")
Are the ES logs available during this time? ---
**Author:** ![tylersmalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tylersmalley/32/8833_2.png) [@tylersmalley](https://discuss.elastic.co/u/tylersmalley)\ **Post date:** [November 16, 2018, 6:12pm UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050/5 "2018-11-16T18:12:20Z")
How many ES nodes in the cluster do you have? ---
**Author:** ![kribor1](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@kribor1](https://discuss.elastic.co/u/kribor1)\ **Post date:** [November 17, 2018, 10:43am UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050/6 "2018-11-17T10:43:26Z")
I have 2 nodes. Aaahhh, checked the elasticsearch logs and found: org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]; I've had similar issues before so I removed read-only from all indices: `curl -X PUT http://localhost:9200/_all/_settings -H 'Content-Type: application/json' -d'{ "index.blocks.read_only_allow_delete" : false } }'` And now kibana finishes migration! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 15, 2018, 10:52am UTC](https://discuss.elastic.co/t/solved-kibana-crashes-on-reindex-after-6-5-0-upgrade/157050/7 "2018-12-15T10:52:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.