# Elasticsearch read-only after upgrade (6.2.0 -\> 6.3.0) **URL:** \ **Category:** Elasticsearch\ **Created:** [June 19, 2018, 1:03pm UTC](https://discuss.elastic.co/t/elasticsearch-read-only-after-upgrade-6-2-0-6-3-0/136496 "2018-06-19T13:03:48Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![jarnold](https://avatars.discourse-cdn.com/v4/letter/j/df788c/32.png) [@jarnold](https://discuss.elastic.co/u/jarnold)\ **Post date:** [June 19, 2018, 1:03pm UTC](https://discuss.elastic.co/t/elasticsearch-read-only-after-upgrade-6-2-0-6-3-0/136496/1 "2018-06-19T13:03:48Z")
I upgraded elasticsearch, kibana and logstash this morning, which seems to have caused the database to become read-only, as evidenced by the following log lines; ``` [2018-06-19T13:51:37,700][ERROR][o.e.x.w.e.ExecutionService] [elastic01] could not store triggered watch with id [ansAPnhIS3atR9t5kZBBkA_elasticsearch_cluster_status_5b19d0ab-a936-4fa5-852f-05fbcd4b09e7-2018-06-19T12:51:37.698Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] [2018-06-19T13:51:37,700][ERROR][o.e.x.w.e.ExecutionService] [elastic01] could not store triggered watch with id [ansAPnhIS3atR9t5kZBBkA_logstash_version_mismatch_aec810fd-095f-4212-9eef-e264f1386cd6-2018-06-19T12:51:37.698Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] [2018-06-19T13:51:37,700][ERROR][o.e.x.w.e.ExecutionService] [elastic01] could not store triggered watch with id [ansAPnhIS3atR9t5kZBBkA_elasticsearch_nodes_dfd70867-8209-4dfe-baca-7de432db4392-2018-06-19T12:51:37.698Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] [2018-06-19T13:51:37,700][ERROR][o.e.x.w.e.ExecutionService] [elastic01] could not store triggered watch with id [ansAPnhIS3atR9t5kZBBkA_xpack_license_expiration_0b1a0f65-6ff7-454b-85d4-0aed4b4a1abb-2018-06-19T12:51:37.698Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] [2018-06-19T13:51:37,700][ERROR][o.e.x.w.e.ExecutionService] [elastic01] could not store triggered watch with id [ansAPnhIS3atR9t5kZBBkA_elasticsearch_version_mismatch_e62ef3e7-3aaf-492e-b716-9b4dd052352f-2018-06-19T12:51:37.698Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] [2018-06-19T13:51:37,701][ERROR][o.e.x.w.e.ExecutionService] [elastic01] could not store triggered watch with id [ansAPnhIS3atR9t5kZBBkA_kibana_version_mismatch_bda4cae4-728b-4f30-a344-7ff5e7a9730a-2018-06-19T12:51:37.698Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] ``` I'm seeing quite a few "version mismatch" errors which I find strange - yum info tells me all components are running on 6.3.0 so i'm not sure how they could be mismatched. ``` [root@elastic01 ~]# yum info elasticsearch Loaded plugins: ulninfo, versionlock Installed Packages Name : elasticsearch Arch : noarch Version : 6.3.0 Release : 1 Size : 136 M Repo : installed From repo : elasticsearch-6.x Summary : Elasticsearch is a distributed RESTful search engine built for the cloud. Reference documentation can be found at : https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html and the 'Elasticsearch: The Definitive Guide' book can be found at : https://www.elastic.co/guide/en/elasticsearch/guide/current/index.html URL : https://www.elastic.co/ License : Elastic License Description : Elasticsearch subproject :distribution:packages [root@elastic01 ~]# yum info kibana Loaded plugins: ulninfo, versionlock Installed Packages Name : kibana Arch : x86_64 Version : 6.3.0 Release : 1 Size : 482 M Repo : installed From repo : elasticsearch-6.x Summary : Explore and visualize your Elasticsearch data URL : https://www.elastic.co License : Elastic License Description : Explore and visualize your Elasticsearch data [root@elastic01 ~]# yum info logstash Loaded plugins: ulninfo, versionlock Installed Packages Name : logstash Arch : noarch Epoch : 1 Version : 6.3.0 Release : 1 Size : 238 M Repo : installed From repo : elasticsearch-6.x Summary : An extensible logging pipeline URL : http://www.elasticsearch.org/overview/logstash/ License : Elastic License Description : An extensible logging pipeline ``` I followed [these](https://www.elastic.co/guide/en/elasticsearch/reference/current/rolling-upgrades.html) instructions to upgrade elasticsearch. Kibana and logstash were stopped, updated, and restarted as per their instructions. I've had this issue before when running out of disk space, currently I have 65% of disk space free so it's not due to that. On a related note, is there any way to automatically make elasticsearch writeable after running out of disk space? Clearing space on the disk does not make a difference, I must run a command manually in order to remove the read-only flag. Checking the health of my indices shows them as "Yellow" as follows; ``` [root@elastic01 ~]# curl -X GET "localhost:9200/_cat/health" 1529413219 14:00:19 elasticstack yellow 1 1 11 11 0 0 5 0 - 68.8% ``` The progress is going up, albeit very slowly. Between 9AM this morning and now (2PM) it has climbed only 4%. This is a single node cluster, with only around 1.5GB of data currently stored - The time this node is taking to come back up seems excessive, but i'm not sure where to begin looking for a problem with this. I can confirm that entering the following into kibana's console resolves the issue; PUT .kibana/\_settings { "index": { "blocks": { "read\_only\_allow\_delete": "false" } } } This is a pain though, I would rather avoid having to run this manually any time elastic becomes read only. I'd rather avoid it becoming read only at all! Thanks ---
**Author:** ![FrankyFire](https://avatars.discourse-cdn.com/v4/letter/f/ea5d25/32.png) [@FrankyFire](https://discuss.elastic.co/u/FrankyFire)\ **Post date:** [July 2, 2018, 1:28pm UTC](https://discuss.elastic.co/t/elasticsearch-read-only-after-upgrade-6-2-0-6-3-0/136496/2 "2018-07-02T13:28:32Z")
Hey there, I can confirm that issue with my ES 6.3.0 Test environment. For me, the problems began when installing X-Pack and trying to integrate Logstash Pipeline Management (without having the necessary license, as I found out later). But that can also be a coincidence. Workaround is the same as you posted. There's also enough disk space left as in your case. Would be nice, if someone could take a closer look into this. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 30, 2018, 1:28pm UTC](https://discuss.elastic.co/t/elasticsearch-read-only-after-upgrade-6-2-0-6-3-0/136496/3 "2018-07-30T13:28:51Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.