# Elasticsearch could not store watch with watch id **URL:** \ **Category:** Elasticsearch\ **Created:** [February 8, 2019, 11:52am UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619 "2019-02-08T11:52:06Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 8, 2019, 11:52am UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/1 "2019-02-08T11:52:06Z")
Hi, Am using ELK version 6.5.4. Running with trial license. I am getting the below ERROR message. could not store triggered watch with id [\_d6Y1y0bQ4mypPMdh3YZzg\_xpack\_license\_expiration\_89c0aa03-0208-400b-a351-db9cf57c2d2b-2019-02-08T11:47:17.631Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] Not storing logs from Logstash. Can anyone help? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 8, 2019, 11:58am UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/2 "2019-02-08T11:58:32Z")
That typically means that you are running out of disk space and that Elasticsearch has put the indices in read-only mode as the [flood stage watermark has been exceeded](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/disk-allocator.html). ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 8, 2019, 12:05pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/3 "2019-02-08T12:05:16Z")
Have more than enough disk space. Space not an issue. Thanks ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 8, 2019, 12:17pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/4 "2019-02-08T12:17:51Z")
Sorry. I missed the first part of the message which indicates that you trial license may have expired. ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 8, 2019, 12:21pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/5 "2019-02-08T12:21:06Z")
Recently enabled it. 1 month didnt complete. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 8, 2019, 12:22pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/6 "2019-02-08T12:22:27Z")
Check what the [get license API](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/get-license.html) shows. ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 8, 2019, 12:23pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/7 "2019-02-08T12:23:40Z")
Thanks for your quick reply GET /\_xpack/license { "license" : { "status" : "active", "uid" : "dbe10339-7784-452e-980e-d3990d39087b", "type" : "trial", "issue\_date" : "2019-01-30T09:24:55.548Z", "issue\_date\_in\_millis" : 1548840295548, "expiry\_date" : "2019-03-01T09:24:55.548Z", "expiry\_date\_in\_millis" : 1551432295548, "max\_nodes" : 1000, "issued\_to" : "elasticsearch", "issuer" : "elasticsearch", "start\_date\_in\_millis" : -1 } } ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 8, 2019, 12:24pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/8 "2019-02-08T12:24:16Z")
Then I am not sure what is wrong... ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 8, 2019, 12:26pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/9 "2019-02-08T12:26:13Z")
@Christian_Dahlqvist : Thanks for checking this. May I know who else can help on this? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [February 8, 2019, 12:49pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/10 "2019-02-08T12:49:27Z")
I would recommend verifying that the data path Elasticsearch uses has enough space. If you are on linux, post the output of `df -k` here so we can see. ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 8, 2019, 1:03pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/11 "2019-02-08T13:03:38Z")
/etc/elasticsearch/elasticsearch.yml path.data: /datavg/elasticsearch/data Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg.datavg-lv.datavg 99G 6.8G 87G 8% /datavg ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 8, 2019, 1:08pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/12 "2019-02-08T13:08:22Z")
When restarting getting this exception .................. .................. [2019-02-08T12:01:50,651][INFO][o.e.n.Node] [Az0RWz6] started [2019-02-08T12:01:53,285][INFO][o.e.m.j.JvmGcMonitorService] [Az0RWz6] [gc][8] overhead, spent [297ms] collecting in the last [1s] [2019-02-08T12:01:53,376][WARN][r.suppressed] [Az0RWz6] path: /.kibana/doc/config%3A6.5.4, params: {index=.kibana, id=config:6.5.4, type=doc} org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE\_UNAVAILABLE/1/state not recovered / initialized]; at org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(ClusterBlocks.java:166) ~[elasticsearch-6.5.4.jar:6.5.4] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.checkGlobalBlock(TransportSingleShardAction.java:122) ~[elasticsearch-6.5.4.jar:6.5.4] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:156) ~[elasticsearch-6.5.4.jar:6.5.4] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction.(TransportSingleShardAction.java:140) ~[elasticsearch-6.5.4.jar:6.5.4] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:97) ~[elasticsearch-6.5.4.jar:6.5.4] at org.elasticsearch.action.support.single.shard.TransportSingleShardAction.doExecute(TransportSingleShardAction.java:61) ~[elasticsearch-6.5.4.jar:6.5.4] at org.elasticsearch.action.support.TransportAction.doExecute(TransportAction.java:143) ~[elasticsearch-6.5.4.jar:6.5.4] at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:167) ~[elasticsearch-6.5.4.jar:6.5.4] ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 11, 2019, 6:14am UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/13 "2019-02-11T06:14:44Z")
Some more information related to this. ``` { "id" : "add_to_alerts_index", "type" : "index", "status" : "failure", "error" : { "root_cause" : [ { "type" : "cluster_block_exception", "reason" : "blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];" } ], "type" : "cluster_block_exception", "reason" : "blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];" } ``` ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [February 11, 2019, 7:38am UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/14 "2019-02-11T07:38:24Z")
> [@Kanthasamyraja](#): > > ```plaintext > "reason" : "blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];" > > ``` This means that someone or something has applied the [`index.blocks.read_only_allow_delete` block](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html) to this index. This is applied automatically when [running out of disk space](https://www.elastic.co/guide/en/elasticsearch/reference/6.6/disk-allocator.html), and this is usually where this block comes from. There will be messages in the logs indicating that this has happened if so. The block is not removed when disk usage drops, so you have to manually remove it as per the reference docs. You can also apply this block manually via the index settings API, so if you don't think you ran out of disk space then that's where it came from. ---
**Author:** ![Kanthasamyraja](https://avatars.discourse-cdn.com/v4/letter/k/ed655f/32.png) [@Kanthasamyraja](https://discuss.elastic.co/u/Kanthasamyraja)\ **Post date:** [February 11, 2019, 4:41pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/15 "2019-02-11T16:41:30Z")
Elasticsearch keep printing below line in log could not store triggered watch with id [\_d6Y1y0bQ4mypPMdh3YZzg\_logstash\_version\_mismatch\_1e150970-84b7-4d14-b2c3-9887a73e038b-2019-02-11T12:12:47.491Z]: [ClusterBlockException[blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];]] But now Indexes are created and app logs are available in Kibana. How to clear this error message? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [February 11, 2019, 6:27pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/16 "2019-02-11T18:27:46Z")
> [@DavidTurner](#): > > The block is not removed when disk usage drops, so you have to manually remove it as per the reference docs. See links above. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 11, 2019, 6:27pm UTC](https://discuss.elastic.co/t/elasticsearch-could-not-store-watch-with-watch-id/167619/17 "2019-03-11T18:27:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.