# How to delete UNASSIGNED .watches?
**URL:** https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091
**Category:** Elasticsearch
**Created:** [December 4, 2017, 6:40am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091 "2017-12-04T06:40:44Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@tjliu](https://discuss.elastic.co/u/tjliu)
#### Post date: [December 4, 2017, 6:40am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/1 "2017-12-04T06:40:44Z")
Hi,
The status of my cluster is RED. It seems it will be green if I delete 2 UNASSIGNED ".watches" from the following log. How can I fix this? Thanks.
> Blockquote
> $ curl -XGET '[http://xxx:9200/\_cat/shards](http://xxx:9200/_cat/shards)'
> .watcher-history-3-2017.09.15 0 r STARTED 7200 5.3mb 10.99.30.38 node1
> .monitoring-es-6-2017.11.29 0 r STARTED 741817 522.9mb 10.99.30.34 node2
> .monitoring-es-6-2017.11.29 0 p STARTED 741817 520.7mb 10.99.23.41 node2
> .watches 0 p UNASSIGNED
> .watches 0 r UNASSIGNED
> .watcher-history-3-2017.10.06 0 p STARTED 6537 5.6mb 10.99.30.34 nod3
> Blockquote
> $ curl -XGET 'xxx:9200/\_cluster/health?pretty'
> {
> "cluster\_name" : "cluster",
> "status" : "red",
> "timed\_out" : false,
> "number\_of\_nodes" : 6,
> "number\_of\_data\_nodes" : 6,
> "active\_primary\_shards" : 72,
> "active\_shards" : 139,
> "relocating\_shards" : 0,
> "initializing\_shards" : 0,
> "unassigned\_shards" : 2,
> "delayed\_unassigned\_shards" : 0,
> "number\_of\_pending\_tasks" : 1,
> "number\_of\_in\_flight\_fetch" : 0,
> "task\_max\_waiting\_in\_queue\_millis" : 0,
> "active\_shards\_percent\_as\_number" : 98.58156028368793
> }
> Blockquote
> $ curl -XGET '[http://xxx:9200/\_cluster/allocation/explain?pretty](http://xxx:9200/_cluster/allocation/explain?pretty)'
> {
> "index" : ".watches",
> "shard" : 0,
> "primary" : true,
> "current\_state" : "unassigned",
> "unassigned\_info" : {
> "reason" : "CLUSTER\_RECOVERED",
> "at" : "2017-11-28T14:31:53.291Z",
> "last\_allocation\_status" : "no\_valid\_shard\_copy"
> },
> "can\_allocate" : "no\_valid\_shard\_copy",
> "allocate\_explanation" : "cannot allocate because all found copies of the shard are either stale or corrupt",
> "node\_allocation\_decisions" : [
> {
> "node\_id" : "xxxplqMrRjWS78Xxxxxxxx",
> "node\_name" : "node1",
> "transport\_address" : "xxx:9300",
> "node\_decision" : "no",
> "store" : {
> "in\_sync" : true,
> "allocation\_id" : "xxxxxleuSGaMc4m2GVxxxx",
> "store\_exception" : {
> "type" : "corrupt\_index\_exception",
> "reason" : "failed engine (reason: [refresh failed]) (resource=preexisting\_corruption)",
> "caused\_by" : {
> "type" : "i\_o\_exception",
> "reason" : "failed engine (reason: [refresh failed])",
> "caused\_by" : {
> "type" : "corrupt\_index\_exception",
> "reason" : "compound sub-files must have a valid codec header and footer: file is too small (0 bytes) (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/path/to/data/nodes/0/indices/xxxx61AmRUW2rwkR8Uxxxx/0/index/\_1d31.nvd")))"
> }
> }
> }
> }
> },
> {
> "node\_id" : "xxxxElcURkqsQYG\_hKxxxx",
> "node\_name" : "node2",
> "transport\_address" : "xxx:9300",
> "node\_decision" : "no",
> "store" : {
> "found" : false
> }
> },
---
### Author:  [@JKhondhu](https://discuss.elastic.co/u/JKhondhu)
#### Post date: [December 5, 2017, 8:38am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/2 "2017-12-05T08:38:23Z")
Hi,
Before deleting or moving the corrupt shards out of the way, can you cat or strings on them to check the content.
Did you recently face any disk or power issues?
---
### Author:  [@tjliu](https://discuss.elastic.co/u/tjliu)
#### Post date: [December 6, 2017, 5:44am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/3 "2017-12-06T05:44:51Z")
This is the cat result. This node already exclude from cluster by re-routing. And the data seems very old to me. It can be deleted.
I dont't know the reason. Maybe because of power issue.
> Blockquote
> curl -XGET '[http://xxx:9200/\_cat/indices/.watches?pretty](http://xxx:9200/_cat/indices/.watches?pretty)'
> red open .watches 0aRC61AmRUW2rwkR8xxxxx 1 1
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [December 6, 2017, 8:57am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/4 "2017-12-06T08:57:31Z")
the reason seems to indicate that a file has a size of zero, even though it should have contents. This indicates an issue with the files on disk (this can be an clearing of the file by an admin, a corrupt disk or a bug in lucene, an unfinished recovery that was aborted before the other node completely vanished).
if you have more indications what happened due to having more logs, I'd be happy to take a look.
Can you explain what you mean with 'This node already exclude from cluster by re-routing' - has this node split away from the cluster and still is a master? If thats the case, then you should check your cluster configuration, especially with regards to the minimum master nodes setting.
Apart from that, if you dont need that index, you can safely delete it. I suppose you are not using watcher, but only monitoring - monitoring in turn creates watches - and they get recreated over time.
Hope this helps!
---
### Author:  [@tjliu](https://discuss.elastic.co/u/tjliu)
#### Post date: [December 8, 2017, 4:10am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/5 "2017-12-08T04:10:02Z")
Hi spinscale,
About 'This node already exclude from cluster by re-routing', I mean I don't want this node by removing it from cluster just use following command:
> Blockquote
> curl -XPUT '[http://xxx:9200/\_cluster/settings?pretty](http://xxx:9200/_cluster/settings?pretty)' -d '
> {
> "transient" : {
> "cluster.routing.allocation.exclude.\_name" : "this node"
> }
> }'
You said I could safely delete it. May I ask how? I used this command and it has error:
> Blockquote
> curl -XDELETE '[http://xxx:9200/.watches/?pretty](http://xxx:9200/.watches/?pretty)'
> {"error":"This endpoint is not supported for DELETE on .watches index.","status":400}%
---
### Author:  [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)
#### Post date: [December 8, 2017, 4:20am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/6 "2017-12-08T04:20:36Z")
Hi @tjliu,
I think you try this query in kibana Dev Tool your cluster is come in green and unassigned shards will be removed.
PUT /\_settings
{
"index": {
"number\_of\_replicas" : 0
}
}
---
### Author:  [@tjliu](https://discuss.elastic.co/u/tjliu)
#### Post date: [December 8, 2017, 7:30am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/7 "2017-12-08T07:30:44Z")
Hi @[Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)
I tried your command in kibana Dev Tool, but I got timeout message. Maybe because of I'm still doing big bulk indexing.
> Blockquote
> {
> "statusCode": 504,
> "error": "Gateway Timeout",
> "message": "Client request timeout"
> }
But I tried this command to set number\_of\_replicas to 0 of index ".watches". Glad it worked: one UNASSIGNED ".watches" were removed. I still got one left. How to remove this last one?
> Blockquote
> curl -XPUT '[http://xxx:9200/.watches/\_settings/?pretty](http://xxx:9200/.watches/_settings/?pretty)' -d '
> {
> "index": {
> "number\_of\_replicas" : 0
> }
> }'
> Blockquote
> curl -XGET 'xxx:9200/\_cat/shards?h=index,shard,prirep,state,unassigned.reason' | grep UNASSIGNED
> .watches 0 p UNASSIGNED CLUSTER\_RECOVERED
> Blockquote
> curl -XDELETE '[http://xxx:9200/.watches/?pretty](http://xxx:9200/.watches/?pretty)'
> {"error":"This endpoint is not supported for DELETE on .watches index.","status":400}%
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [December 8, 2017, 9:48am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/8 "2017-12-08T09:48:00Z")
try to delete by running `curl -X DELETE http://xxx:9200/.watches*`
---
### Author:  [@tjliu](https://discuss.elastic.co/u/tjliu)
#### Post date: [December 8, 2017, 10:54am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/9 "2017-12-08T10:54:55Z")
hi @spinscale
Thanks a lot. It worked. My cluster is finally green. 😀
---
### Author:  [@Krunal\_kalaria](https://discuss.elastic.co/u/Krunal_kalaria)
#### Post date: [December 8, 2017, 11:05am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/10 "2017-12-08T11:05:13Z")
@spinscale good solution done by you. 🙂
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 5, 2018, 11:05am UTC](https://discuss.elastic.co/t/how-to-delete-unassigned-watches/110091/11 "2018-01-05T11:05:13Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.