# Lost or corrupt elasticsearch indicies

**URL:** <https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045>\
**Category:** Elasticsearch\
**Created:** [May 26, 2016, 10:36am UTC](https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045 "2016-05-26T10:36:46Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![stevebeattie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stevebeattie/32/9228_2.png) [@stevebeattie](https://discuss.elastic.co/u/stevebeattie)\
**Post date:** [May 26, 2016, 10:36am UTC](https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045/1 "2016-05-26T10:36:46Z")

</div>

Hi,  
I have an ELK server running that monitors our production IHS (Apache) web servers. The ELK server was running well until today when I think that elasticsearch ran out of space on the file system used to store the data. Installed versions are as follows

```auto
# rpm -qa | grep -E "logstash|elasticsearch"
logstash-2.1.1-1.noarch
elasticsearch-2.1.1-1.noarch
# cat /etc/redhat-release
CentOS release 6.7 (Final)
```

I have manually removed the indices from elasticsearch as my normal method of using curl -XDELETE wouldn't work.

Now when I go to kibana I get the following message:-

```auto
Warning No default index pattern. You must select or create one to continue.
```

There's a stack trace in the elasticsearch.log file as follows (truncated to keep message within limits):-

```auto
[2016-05-26 11:33:06,127][INFO][rest.suppressed] /logstash-*/_mapping/field/* Params: {index=logstash-*, allow_no_indices=false, include_defaults=true, _=1464258787353, fields=*, ignore_unavailable=false}
[logstash-*] IndexNotFoundException[no such index]
        at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver$WildcardExpressionResolver.resolve(IndexNameExpressionResolver.java:636)
        at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:133)
        at org.elasticsearch.cluster.metadata.IndexNameExpressionResolver.concreteIndices(IndexNameExpressionResolver.java:77)
        at org.elasticsearch.action.admin.indices.mapping.get.TransportGetFieldMappingsAction.doExecute(TransportGetFieldMappingsAction.java:57)
        at org.elasticsearch.action.admin.indices.mapping.get.TransportGetFieldMappingsAction.doExecute(TransportGetFieldMappingsAction.java:40)
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:70)
        at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:347)
        at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52)
        at org.elasticsearch.rest.BaseRestHandler$HeadersAndContextCopyClient.doExecute(BaseRestHandler.java:83)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:347)
        at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.execute(AbstractClient.java:1183)
        at org.elasticsearch.client.support.AbstractClient$IndicesAdmin.getFieldMappings(AbstractClient.java:1383)
        at org.elasticsearch.rest.action.admin.indices.mapping.get.RestGetFieldMappingAction.handleRequest(RestGetFieldMappingAction.java:66)
        at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:54)
        at org.elasticsearch.rest.RestController.executeHandler(RestController.java:207)
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:166)
        at org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:128)
        at org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:86)
        at org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:348)
        at org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:63)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.elasticsearch.http.netty.pipelining.HttpPipeliningHandler.messageReceived(HttpPipeliningHandler.java:60)
        at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)

```

So, how can I recover my ELK server?

Thanks in advance, Steve.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [May 26, 2016, 10:04pm UTC](https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045/2 "2016-05-26T22:04:23Z")

</div>

If you have deleted the indices manually what shows in `_cat/indices`?

---

<div class="post-metadata">

**Author:** ![stevebeattie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stevebeattie/32/9228_2.png) [@stevebeattie](https://discuss.elastic.co/u/stevebeattie)\
**Post date:** [May 27, 2016, 6:35am UTC](https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045/3 "2016-05-27T06:35:26Z")

</div>

Hi,  
Thanks for the reply, is that a call I need to make via curl or is it on the server somewhere?

Thanks, Steve.

---

<div class="post-metadata">

**Author:** ![stevebeattie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stevebeattie/32/9228_2.png) [@stevebeattie](https://discuss.elastic.co/u/stevebeattie)\
**Post date:** [May 27, 2016, 10:33am UTC](https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045/4 "2016-05-27T10:33:19Z")

</div>

Hi,  
Ok, I have the following in \_cat/indices:-

```auto

yellow open filebeat-2016.05.27 5 1 1516795 0 2.4gb 2.4gb 
yellow open filebeat-2016.05.26 5 1 6644009 0 10.3gb 10.3gb 
yellow open topbeat-2016.05.27 5 1 6313099 0 1.4gb 1.4gb 
yellow open topbeat-2016.05.26 5 1 10750583 0 2.5gb 2.5gb 
yellow open packetbeat-2016.05.26 5 1 9833834 0 3.5gb 3.5gb 
yellow open packetbeat-2016.05.27 5 1 2186287 0 841.7mb 841.7mb 
yellow open .kibana 1 1 132 0 99kb 99kb 
```

Thanks, Steve

---

<div class="post-metadata">

**Author:** ![stevebeattie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stevebeattie/32/9228_2.png) [@stevebeattie](https://discuss.elastic.co/u/stevebeattie)\
**Post date:** [May 27, 2016, 11:43am UTC](https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045/5 "2016-05-27T11:43:31Z")

</div>

An update... I don't know how to explain this, but when I restarted my browser the Kibana pages now work and the exceptions in the elasticsearch log have stopped too. Was something persisted in by the browser?

Anyway - problem solved, or at least has gone away.

Thanks, Steve.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:48pm UTC](https://discuss.elastic.co/t/lost-or-corrupt-elasticsearch-indicies/51045/6 "2017-07-05T22:48:30Z")

</div>


