# Upgrade issue with Elastic Stack 5.6.0, workaround option until fix is available **URL:** \ **Category:** Kibana\ **Created:** [September 14, 2017, 7:27pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595 "2017-09-14T19:27:47Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![marty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marty/32/46178_2.png) [@marty](https://discuss.elastic.co/u/marty)\ **Post date:** [September 14, 2017, 7:27pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/1 "2017-09-14T19:27:47Z")
Since the release of Elastic Stack 5.6.0, we’ve discovered an issue during the upgrade process to this version. ## Who is affected Anyone upgrading a cluster that has indexes created with Elasticsearch 2.x or prior. This does not affect users installing a fresh 5.6 cluster nor those who started their Elastic journey with the 5.x releases. ## Symptom The problem is most visible through Kibana. After attempting to login to Kibana post-upgrade, the following error appears: ![1d38a818-20d1-40ca-863d-e75f815c978b](https://us1.discourse-cdn.com/elastic/original/3X/c/a/ca72f30b051bfa57a8b93d090f27820a176d04a7.png) It may also manifests as the following error: ![8dcb4f04-8dc2-4b8d-92d7-d823764595b3](https://us1.discourse-cdn.com/elastic/original/3X/2/6/26977d5aefaf86205774280bb2e25f55608c6c23.jpeg) Or if [X-Pack](https://www.elastic.co/products/x-pack) is in use: ![99490844-2e09-4ec1-9004-1c536ee46d86](https://us1.discourse-cdn.com/elastic/original/3X/a/4/a4f55ff66a9c37e02189c4f24ec38347d2623940.png) In any case, the Elasticsearch cluster itself (including all of your data in Elasticsearch) is not affected and correctly reports itself as in Green status. ## Mitigation We have created a hotfix script you can use to fix the issues in your cluster. The script must be run on a Linux system with network access to an instance in the impacted Elasticsearch cluster. Alternatively, waiting for 5.6.1 is also an option. **Using the hotfix script:** **1 - Stop Kibana** Stop the existing Kibana process using the appropriate command for your system. For example on a systemd-based Linux distribution installed via our RPM/DEB packages: `sudo systemctl stop kibana` **2 - Download the hotfix script** The patch is available to download [here](https://download.elastic.co/downloads/kibana/8134-patch.sh). You can download directly to your workstation or server with curl, for example: `curl -L -O https://download.elastic.co/downloads/kibana/8134-patch.sh` **3 - Run the hotfix** _Without X-Pack Installed_ If you are not using X-Pack then you need only specify a URL to a node in your cluster, for example: `./8134-patch.sh http://mycluster:9200` If you do not specify a URL, it defaults to [http://localhost:9200](http://localhost:9200) _With X-Pack Installed_ If you are using X-Pack and have the Security enabled, you'll need to pass credentials for a cluster user (assigned to the superuser built-in role), for example: `./8134-patch.sh https://mycluster:9200 elastic:mypassword` **4 - Start Kibana** Start the existing Kibana process using the appropriate command for your system. For example on a systemd-based Linux distribution installed via our RPM/DEB packages: `sudo systemctl start kibana` ## Cause The root cause is a [bug in Elasticsearch](https://github.com/elastic/elasticsearch/issues/26162) we are targeting to fix in 5.6.1, and that release will be imminent. Waiting for 5.6.1 is a viable option if you have not already upgraded. ---
**Author:** ![marty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marty/32/46178_2.png) [@marty](https://discuss.elastic.co/u/marty)\ **Post date:** [September 14, 2017, 7:29pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/2 "2017-09-14T19:29:46Z")
---
**Author:** ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)\ **Post date:** [September 18, 2017, 9:59pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/3 "2017-09-18T21:59:58Z")
The Elastic Stack 5.6.1 was released today; [https://www.elastic.co/downloads](https://www.elastic.co/downloads) ---
**Author:** ![cutch69](https://avatars.discourse-cdn.com/v4/letter/c/977dab/32.png) [@cutch69](https://discuss.elastic.co/u/cutch69)\ **Post date:** [September 19, 2017, 8:30pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/4 "2017-09-19T20:30:25Z")
I have this same issue but started with a 5.x system Upgrade from 5.5.2 to 5.6.1 ---
**Author:** ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)\ **Post date:** [September 19, 2017, 11:56pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/5 "2017-09-19T23:56:30Z")
@cutch69 I just went through that case and everything worked fine for me. Here's what I did; Installed the full 5.5.2 stack on Ubuntu VM. Includes Elasticsearch, Kibana, Logstash, and 3 of the beats, with X-Pack installed in Elasticsearch, Kibana, and Logstash (but should all work fine without X-Pack). The I upgraded, first Elasticsearch to 5.6.1, started Kibana and made sure things were working. Then I stopped Kibana and upgraded it to 5.6.1 and tested again. I didn't upgrade or restart logstash or the beats since they didn't seem to be relevant to this issue. ```auto root@packer-virtualbox-iso-1501424719:~# history 1 service logstash stop 2 service metricbeat stop 3 service packetbeat stop 4 service filebeat stop 5 service kibana stop 6 service elasticsearch stop 7 /usr/share/elasticsearch/bin/elasticsearch-plugin remove xpack (wrong plugin name) 8 /usr/share/elasticsearch/bin/elasticsearch-plugin remove x-pack 9 cp /etc/elasticsearch/elasticsearch.yml /tmp/ 10 wget http://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.1.deb 11 dpkg -i elasticsearch-5.6.1.deb 12 wget http://artifacts.elastic.co/downloads/packs/x-pack/x-pack-5.6.1.zip 13 pwd 14 /usr/share/elasticsearch/bin/elasticsearch-plugin install -b file:////home/vagrant/x-pack-5.6.1.zip 15 service elasticsearch start 16 service kibana start (I tested here and everything was working) 17 service kibana stop 18 wget http://artifacts.elastic.co/downloads/kibana/kibana-5.6.1-amd64.deb 19 /usr/share/kibana/bin/kibana-plugin remove x-pack 20 dpkg -i kibana-5.6.1-amd64.deb 21 /usr/share/kibana/bin/kibana-plugin install file:////home/vagrant/x-pack-5.6.1.zip 22 service kibana start 23 history ``` ---
**Author:** ![benpolzin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benpolzin/32/12408_2.png) [@benpolzin](https://discuss.elastic.co/u/benpolzin)\ **Post date:** [September 20, 2017, 11:03am UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/6 "2017-09-20T11:03:52Z")
I upgraded to 5.6.0 and hit this issue, but successfully applied the hotfix, `8132-patch.sh`. At this point Kibana was happy and working as expected. I then upgraded ES and Kibana to 5.6.1. Now Kibana is back in a "Red" state with some different errors. ES cluster status is "Green". Is this related to the hotfix patch? Or something completely new? ![kibanared](https://us1.discourse-cdn.com/elastic/original/3X/2/f/2f9f224c7458cd7dc53123c7992e9ce968f0a1e5.png) I also see this show up in Kibana's stdout log over and over. ```auto {"type":"log","@timestamp":"2017-09-20T10:57:28Z","tags":["info","elasticsearch"],"pid":17028,"typeName":"url","typeMapping":{"properties":{"accessCount":{"type":"long"},"accessDate":{"type":"date"},"createDate":{"type":"date"},"url":{"type":"text","fields":{"keyword":{"type":"keyword","ignore_above":2048}}}}},"message":"Adding mappings to kibana index for SavedObject type \"url\""} ``` No timestamps in the stderr log, but here's a snippet from that: ```auto Error while executing search { Error: Request Timeout after 30000ms at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:336:15 at Timeout. (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:365:7) at ontimeout (timers.js:365:14) at tryOnTimeout (timers.js:237:5) at Timer.listOnTimeout (timers.js:207:5) status: undefined, displayName: 'RequestTimeout', message: 'Request Timeout after 30000ms', body: undefined } Unhandled rejection Error: Request Timeout after 30000ms at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:336:15 at Timeout. (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:365:7) at ontimeout (timers.js:365:14) at tryOnTimeout (timers.js:237:5) at Timer.listOnTimeout (timers.js:207:5) Unhandled rejection Error: No Living connections at sendReqWithConnection (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:207:15) at next (/usr/share/kibana/node_modules/elasticsearch/src/lib/connection_pool.js:213:7) at _combinedTickCallback (internal/process/next_tick.js:67:7) at process._tickDomainCallback (internal/process/next_tick.js:122:9) ``` ---
**Author:** ![cutch69](https://avatars.discourse-cdn.com/v4/letter/c/977dab/32.png) [@cutch69](https://discuss.elastic.co/u/cutch69)\ **Post date:** [September 20, 2017, 11:57am UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/7 "2017-09-20T11:57:50Z")
Yeah i figure out what the issue was i had to upgrade both kibana servers. seems odd ---
**Author:** ![Ant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ant/32/33267_2.png) [@Ant](https://discuss.elastic.co/u/Ant)\ **Post date:** [September 20, 2017, 3:20pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/8 "2017-09-20T15:20:57Z")
I have installed 5.6.1 and I'm getting the xpack issue, I've tried to apply the hot fix above with no joy. Is the old patch version specific and there is a seperate issue with 5.6.1 that still needs to be addressed? ---
**Author:** ![benpolzin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benpolzin/32/12408_2.png) [@benpolzin](https://discuss.elastic.co/u/benpolzin)\ **Post date:** [September 21, 2017, 2:21pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/9 "2017-09-21T14:21:44Z")
After the 5.6.1 upgrade (and before that 5.6.0 + [8134-patch.sh](http://8134-patch.sh) workaround) I'm seeing this: ```auto [2017-09-21T09:17:51,091][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [elkelastic3] failed to put mappings on indices [[[.kibana-6/LaISONzyRfGlu2gzSLxvhA]]], type [url] java.lang.IllegalArgumentException: [url] is defined as a field in mapping [url] but this name is already used for an object in other types ``` This appears to be related to the patch + 5.6.1. Anyone know the safe/recommended way to recover from this? ---
**Author:** ![tylersmalley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tylersmalley/32/8833_2.png) [@tylersmalley](https://discuss.elastic.co/u/tylersmalley)\ **Post date:** [September 21, 2017, 6:32pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/10 "2017-09-21T18:32:27Z")
@Ant & @benpolzin , Would you mind separately posting issues outlining the process you went through? Including, what versions you upgrade to, if and when you ran the script, etc. Mention me and I will work with you through it. Thanks and apologies for any inconvenience you are experiencing. ---
**Author:** ![Ant](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ant/32/33267_2.png) [@Ant](https://discuss.elastic.co/u/Ant)\ **Post date:** [September 25, 2017, 2:33pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/11 "2017-09-25T14:33:05Z")
@tylersmalley As I did this on a dev system I've just gone to check with the devs that Elastic is working for them even is kibana isn't and they told me "what are you on about I was using it Friday afternoon". I've done a Ctrl+Shift+R on the kibaba sign in page and I'm no longer getting the error, however in the interests of completeness (and because numpty boy had already typed it out before asking the devs if they saw any issues with the elastic service) this is the process I followed incase it can help anyone else. I upgraded from 5.4.3 straight to 5.6.1 after doing so when I tried to sign into Kibana I was told "Login is currently disabled because the license could not be determined. Please check that Elasticsearch has the X-Pack plugin installed and is reachable, then refresh this page." I then re-registered the license file with no joy after this I came on the forum and saw the above post and downloaded and ran the [8134-patch.sh](http://8134-patch.sh) file but still get the same message about login being disabled. waited 5 days and then had a dev tell me what am I talking about it's working fine... The process I went through to do the 5.4.3 to 5.6.1 upgrade was: disable cluter routing remove x-pack for both elastic and kibana stop elastic and kibana yum install the new version for elastic and kibana install xpack for elastic and kibana start elastic start kibana re-enable cluster routing ---
**Author:** ![marty](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marty/32/46178_2.png) [@marty](https://discuss.elastic.co/u/marty)\ **Post date:** [October 23, 2017, 2:33pm UTC](https://discuss.elastic.co/t/upgrade-issue-with-elastic-stack-5-6-0-workaround-option-until-fix-is-available/100595/12 "2017-10-23T14:33:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.