# ES cluster is down: indexing is not supported on a read-only engine **URL:** \ **Category:** Elasticsearch\ **Created:** [November 2, 2019, 6:27am UTC](https://discuss.elastic.co/t/es-cluster-is-down-indexing-is-not-supported-on-a-read-only-engine/206229 "2019-11-02T06:27:21Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![MosheMallal](https://avatars.discourse-cdn.com/v4/letter/m/ac8455/32.png) [@MosheMallal](https://discuss.elastic.co/u/MosheMallal)\ **Post date:** [November 2, 2019, 6:27am UTC](https://discuss.elastic.co/t/es-cluster-is-down-indexing-is-not-supported-on-a-read-only-engine/206229/1 "2019-11-02T06:27:21Z")
Hi all, we have very sever issue for the past 24H. our ES is down no data is received from logstash into ES. Logstash logs shows unsupported\_operation\_exception: indexing is not supported on a read-only engine. There is no disk space issue and the indices are open (manually indexing is working). Any help will be much appreciated. Thanks, Moshe. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 2, 2019, 8:07am UTC](https://discuss.elastic.co/t/es-cluster-is-down-indexing-is-not-supported-on-a-read-only-engine/206229/2 "2019-11-02T08:07:30Z")
What is the outputof the [cluster health](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/cluster-health.html) and [cluster stats](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/cluster-stats.html) APIs? ---
**Author:** ![MosheMallal](https://avatars.discourse-cdn.com/v4/letter/m/ac8455/32.png) [@MosheMallal](https://discuss.elastic.co/u/MosheMallal)\ **Post date:** [November 2, 2019, 8:12am UTC](https://discuss.elastic.co/t/es-cluster-is-down-indexing-is-not-supported-on-a-read-only-engine/206229/3 "2019-11-02T08:12:41Z")
Hi, Cluster Health: { "cluster\_name" : "credorax-elk-prod", "status" : "green", "timed\_out" : false, "number\_of\_nodes" : 6, "number\_of\_data\_nodes" : 6, "active\_primary\_shards" : 1222, "active\_shards" : 2448, "relocating\_shards" : 2, "initializing\_shards" : 0, "unassigned\_shards" : 0, "delayed\_unassigned\_shards" : 0, "number\_of\_pending\_tasks" : 0, "number\_of\_in\_flight\_fetch" : 0, "task\_max\_waiting\_in\_queue\_millis" : 0, "active\_shards\_percent\_as\_number" : 100.0 } Stats: { "\_nodes" : { "total" : 6, "successful" : 6, "failed" : 0 }, "cluster\_name" : "credorax-elk-prod", "cluster\_uuid" : "IPy2nbl6RZWNev8WfZH\_sg", "timestamp" : 1572682271315, "status" : "green", "indices" : { "count" : 550, "shards" : { "total" : 2448, "primaries" : 1222, "replication" : 1.0032733224222585, "index" : { "shards" : { "min" : 2, "max" : 12, "avg" : 4.450909090909091 }, "primaries" : { "min" : 1, "max" : 6, "avg" : 2.2218181818181817 }, "replication" : { "min" : 1.0, "max" : 5.0, "avg" : 1.0072727272727273 } } }, "docs" : { "count" : 32319209453, "deleted" : 2982642 }, "store" : { "size\_in\_bytes" : 28455475315490 }, "fielddata" : { "memory\_size\_in\_bytes" : 278884020, "evictions" : 0 }, "query\_cache" : { "memory\_size\_in\_bytes" : 362807680, "total\_count" : 3333650, "hit\_count" : 385333, "miss\_count" : 2948317, "cache\_size" : 4313, "cache\_count" : 4313, "evictions" : 0 }, "completion" : { "size\_in\_bytes" : 0 }, "segments" : { "count" : 17285, "memory\_in\_bytes" : 44191871950, "terms\_memory\_in\_bytes" : 39207382835, "stored\_fields\_memory\_in\_bytes" : 3437104248, "term\_vectors\_memory\_in\_bytes" : 0, "norms\_memory\_in\_bytes" : 1224256, "points\_memory\_in\_bytes" : 1487544831, "doc\_values\_memory\_in\_bytes" : 58615780, "index\_writer\_memory\_in\_bytes" : 7270701, "version\_map\_memory\_in\_bytes" : 178843, "fixed\_bit\_set\_memory\_in\_bytes" : 112992, "max\_unsafe\_auto\_id\_timestamp" : 1572645593793, "file\_sizes" : { } } }, "nodes" : { "count" : { "total" : 6, "data" : 6, "coordinating\_only" : 0, "master" : 6, "ingest" : 6 }, "versions" : [ "6.7.2" ], "os" : { "available\_processors" : 96, "allocated\_processors" : 96, "names" : [ { "name" : "Linux", "count" : 6 } ], "pretty\_names" : [ { "pretty\_name" : "CentOS Linux 7 (Core)", "count" : 6 } ], "mem" : { "total\_in\_bytes" : 398111711232, "free\_in\_bytes" : 27745533952, "used\_in\_bytes" : 370366177280, "free\_percent" : 7, "used\_percent" : 93 } }, "process" : { "cpu" : { "percent" : 21 }, "open\_file\_descriptors" : { "min" : 2539, "max" : 3492, "avg" : 2850 } }, "jvm" : { "max\_uptime\_in\_millis" : 2364849, "versions" : [ { "version" : "12.0.1", "vm\_name" : "OpenJDK 64-Bit Server VM", "vm\_version" : "12.0.1+12", "vm\_vendor" : "Oracle Corporation", "count" : 6 } ], "mem" : { "heap\_used\_in\_bytes" : 73909170144, "heap\_max\_in\_bytes" : 194740748288 }, "threads" : 1267 }, "fs" : { "total\_in\_bytes" : 75769646678016, "free\_in\_bytes" : 47260913737728, "available\_in\_bytes" : 47260913737728 }, "plugins" : [ { "name" : "repository-s3", "version" : "6.7.2", "elasticsearch\_version" : "6.7.2", "java\_version" : "1.8", "description" : "The S3 repository plugin adds S3 repositories", "classname" : "org.elasticsearch.repositories.s3.S3RepositoryPlugin", "extended\_plugins" : , "has\_native\_controller" : false } ], "network\_types" : { "transport\_types" : { "security4" : 6 }, "http\_types" : { "security4" : 6 } } } } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 2, 2019, 10:16am UTC](https://discuss.elastic.co/t/es-cluster-is-down-indexing-is-not-supported-on-a-read-only-engine/206229/4 "2019-11-02T10:16:27Z")
Have you checked the status/setting of all indices Logstash could be writing to? Is there anything in the Elasticsearch logs? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [November 2, 2019, 4:05pm UTC](https://discuss.elastic.co/t/es-cluster-is-down-indexing-is-not-supported-on-a-read-only-engine/206229/5 "2019-11-02T16:05:14Z")
I suspect at some point you may have ran out of disk space even though there is disk space now. What happens is elastic goes into read-only mode. Now you will need to reset that read-only mode. [https://www.elastic.co/guide/en/elasticsearch/reference/7.4/disk-allocator.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/disk-allocator.html) [https://www.elastic.co/guide/en/elasticsearch/reference/7.4/index-modules.html](https://www.elastic.co/guide/en/elasticsearch/reference/7.4/index-modules.html) ``` PUT /your_index/_settings { "index.blocks.read_only_allow_delete": null, "index.blocks.read_only": false } ``` Clarification : `false` sets the setting to `false` and `null` remove the setting from the dynamic settings thus cleaning up unnecessary metadata. They have the same result on the index but `null` ends up with less clutter. You may need to run that on multiple indices. Also do you have any frozen indices that you are attempting to index too? If you are trying to index into a frozen index which normally responds with `blocked by: [FORBIDDEN/8/index write (api)]` because by default frozen indices have `index.blocks.write: true`, but if you remove that block then you may get the exception you are seeing. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 30, 2019, 4:05pm UTC](https://discuss.elastic.co/t/es-cluster-is-down-indexing-is-not-supported-on-a-read-only-engine/206229/6 "2019-11-30T16:05:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.