# Elastic search error in application, Error message - " An error occurred while requesting elastic search for data on index 'denmod\_y\_\*' **URL:** \ **Category:** Elasticsearch\ **Created:** [July 12, 2019, 2:51pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261 "2019-07-12T14:51:16Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 12, 2019, 2:51pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/1 "2019-07-12T14:51:17Z")
> Once the application runs & When Elasticsearch for data, it was unable to retrieve. It says error while searching the data. It has two types of error 1) All shards failed error 2) Index not found exception. We have rebooted all the master & Elasticsearch servers but we are still unable to find the solution. Also im new to Elasticsearch. 1. Index not found exception ```auto { "error": { "root_cause": [ { "type": "index_not_found_exception", "reason": "no such index", "resource.type": "index_or_alias", "resource.id": "denmod_y_104242", "index_uuid": "_na_", "index": "denmod_y_104242" } ], "type": "index_not_found_exception", "reason": "no such index", "resource.type": "index_or_alias", "resource.id": "denmod_y_104242", "index_uuid": "_na_", "index": "denmod_y_104242" }, "status": 404 } ``` 1. ```auto { "error": { "root_cause": [ ], "type": "search_phase_execution_exception", "reason": "all shards failed", "phase": "query", "grouped": true, "failed_shards": [ ], "caused_by": { "type": "no_shard_available_action_exception", "reason": null, "index_uuid": "Qtybg8VFStaOKuDkwJ00RA", "shard": "5", "index": "en_y_77" } }, "status": 503 } ``` 1. For first error, we need the shards to be allocated or assigned 2) For second error, index error to be rectified. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 12, 2019, 3:04pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/2 "2019-07-12T15:04:27Z")
What is the output of: ```auto GET / GET /_cat/nodes?v GET /_cat/health?v GET /_cat/indices?v # Optionally GET /_cat/shards?v ``` If some outputs are too big, please share them on [gist.github.com](http://gist.github.com) and link them here. ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 15, 2019, 7:51am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/3 "2019-07-15T07:51:12Z")
Please find the below output: ## GET /\_cat/nodes?v ip heap.percent ram.percent cpu load\_1m load\_5m load\_15m node.role master name 168.124.25.122 19 67 0 di - elk-denmod-6 168.124.54.142 26 68 0 di - elk-denmod-5 168.124.29.126 25 68 0 di - elk-denmod-3 168.124.170.244 25 68 0 di - elk-denmod-4 168.124.25.140 13 66 0 di - elk-denmod-1 168.124.147.161 36 70 24 m \* elk-denmod-web 168.124.29.129 24 67 1 di - elk-denmod-2 ## GET /\_cat/health?v epoch timestamp cluster status node.total node.data shards pri relo init unassign pending\_tasks max\_task\_wait\_time active\_shards\_percent 1563176654 09:44:14 elk-denmod red 7 6 20 10 0 0 596 0 - 3.2% ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 15, 2019, 9:53am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/4 "2019-07-15T09:53:15Z")
Please [read this](https://discuss.elastic.co/t/about-the-elasticsearch-category/21) about how to format. Could you also provide the other outputs I asked for? ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 15, 2019, 10:00am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/5 "2019-07-15T10:00:36Z")
We don't have kibana to give the format like that, we use only cerebro tool elastic search. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 15, 2019, 11:57am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/6 "2019-07-15T11:57:16Z")
And? Does it prevent you from running the queries I asked and format the result? ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 17, 2019, 7:27am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/7 "2019-07-17T07:27:51Z")
please find the below github link for the above outputs. [GitHub\_Link](https://gist.github.com/josephraj08/36b67c5f0bc78059f3f033f38e0d6718) ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [July 17, 2019, 8:06am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/8 "2019-07-17T08:06:49Z")
We can see that a lot of shards are unassigned. Which explains this red state and why you can't really search. You have apparently `596` missing shards. Could you run: ```auto GET /_cluster/pending_tasks GET /_cluster/allocation/explain?include_disk_info=true ``` ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 17, 2019, 9:50am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/9 "2019-07-17T09:50:18Z")
`GET /_cluster/pending_tasks --> No tasks` `GET /_cluster/allocation/explain?include_disk_info=true` > ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 26, 2019, 12:02pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/10 "2019-07-26T12:02:34Z")
Can you please help us, as we are waiting for your reply. How to clear the 596 unassigned shards. ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [July 29, 2019, 12:54am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/11 "2019-07-29T00:54:17Z")
Are you just trying to clear the unassigned shards or recover the data (or whatever is left of it)? The error message is quite explicit what the problem is: `"can_allocate":"no_valid_shard_copy","allocate_explanation":"cannot allocate because all found copies of the shard are either stale or corrupt"` How did you get into that situation in the first place — did you roll back an upgrade after having some nodes up and running in the new version already? You could look into cluster reroute and specifically the [`allocate_stale_primary` section](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-reroute.html#_forced_allocation_on_unrecoverable_errors). This might cause data loss as described in the docs and you will need to set the `accept_data_loss` flag explicitly to acknowledge that. The docs don't have an example so that nobody blindly copies a destructive command; if you need help with the right query, ask again but only after being sure that this is where you want to go. ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 29, 2019, 11:57am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/12 "2019-07-29T11:57:13Z")
We need to recover the data. Could you please provide us the query for getting the data. ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [July 29, 2019, 12:45pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/13 "2019-07-29T12:45:07Z")
**This can cause data loss — use with caution!** The command should be (tested on 7.2): ``` POST /_cluster/reroute { "commands": [ { "allocate_stale_primary": { "accept_data_loss": true, "index": "", "shard": "", "node": "" } } ] } ``` BTW you've avoided the question how you got to this problem, so I'm taking a bit of a wild guess here. There might be other solutions, but without knowing the background it's impossible to say. ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 29, 2019, 1:00pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/14 "2019-07-29T13:00:08Z")
Actually we don't want to lose data in this case, is there any possible way to recover the data. But in the above reply you said this can cause data loss. How you got this problem ? We also not sure for this query. ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [July 29, 2019, 2:43pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/15 "2019-07-29T14:43:41Z")
> [@Joseph\_Raj](#): > > Actually we don't want to lose data in this case, is there any possible way to recover the data. If you have taken a snapshot before I'd restore that. Otherwise I don't see many alternatives (especially without knowing what happened) and the query above might cause data loss. ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 29, 2019, 3:05pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/16 "2019-07-29T15:05:45Z")
> [@xeraa](#): > > can\_allocate":"no\_valid\_shard\_copy","allocate\_explanation":"cannot allocate because all found copies of the shard are either stale or corrupt" Thanks for the query, we initially started with 3 index allocation & it was successful without any data loss. Also we could see the shards are getting reduced. ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 29, 2019, 3:08pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/17 "2019-07-29T15:08:15Z")
I'm not getting on this quote, "If you have taken snapshot before i'd restore that". You meant to say before getting the data loss, i need to take some snapshots. If so, what is the query i need to execute & then i can take snap for restore ? ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [July 29, 2019, 3:10pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/18 "2019-07-29T15:10:18Z")
We have also faced another error on some indices. Please check the below error : `cannot allocate because a previous copy of the primary shard existed but can no longer be found on the nodes in the cluster` Please help us on the above error to fix the same. ---
**Author:** ![xeraa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/xeraa/32/48181_2.png) [@xeraa](https://discuss.elastic.co/u/xeraa)\ **Post date:** [July 29, 2019, 3:26pm UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/19 "2019-07-29T15:26:23Z")
If it's important data, I would have hoped that you took regular snapshots and could restore those. Now it's too late. `cannot allocate because a previous copy of the primary shard existed but can no longer be found on the nodes in the cluster`: Something is very wrong. - Did you lose 2 or more nodes of your cluster (with `replica: 1` it would need to be at least 2)? - Did you delete the data directory (partially) on some nodes or were multiple drives corrupted? - Did something happen to your master node if there was a single one and it lost (part of) the cluster state? Anyway, if you haven't taken a backup before the problems started: Try to recover what you can and make sure you have 3 master eligible nodes (with the correct settings for the quorum — depends on your version). Take snapshots in the future if the data should not be lost. ---
**Author:** ![Joseph\_Raj](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joseph_raj/32/46850_2.png) [@Joseph\_Raj](https://discuss.elastic.co/u/Joseph_Raj)\ **Post date:** [August 1, 2019, 6:40am UTC](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261/20 "2019-08-01T06:40:47Z")
Is there anyway to recover the data for the below error i shown. `cannot allocate because a previous copy of the primary shard existed but can no longer be found on the nodes in the cluster` 1. If we can recover, please provide the query. 2. If not, why we can't recover. please explain. As i'm new to this elastic search. [Next page](https://discuss.elastic.co/t/elastic-search-error-in-application-error-message-an-error-occurred-while-requesting-elastic-search-for-data-on-index-denmod-y/190261.md?page=2)