# java.io.IOException: failed to obtain in-memory shard lock
**URL:** https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375
**Category:** Elasticsearch
**Created:** [September 5, 2018, 10:52am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375 "2018-09-05T10:52:40Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 5, 2018, 10:52am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/1 "2018-09-05T10:52:40Z")
We are intermittently getting the below error in elasticsearch logs. It gets this error after some time, it will be back.
elasticsearch version - "6.3.2"
Real concern in the log
- marking and sending shard failed due to [failed to create shard]
- java.io.IOException: failed to obtain in-memory shard lock
- [70]: failed to obtain shard lock
I have gone through the related blogs which is mentioning this.
so where I need help
1. Why "shard lock" is happening?
2. Is there any data loss when this happens
3. How to track why it is happening. Any command or any suggestion on this so that I get more details on this
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 5, 2018, 10:55am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/2 "2018-09-05T10:55:43Z")
What kind of storage/disk are you using for Elasticsearch?
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 5, 2018, 10:56am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/3 "2018-09-05T10:56:08Z")
We are using RAID0
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 5, 2018, 11:21am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/4 "2018-09-05T11:21:03Z")
Local SSDs? Spinning disks?
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 6, 2018, 3:16am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/5 "2018-09-06T03:16:55Z")
Thanks Christian. It is SSD
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 6, 2018, 6:30am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/6 "2018-09-06T06:30:28Z")
@Christian_Dahlqvist Let me know if you need any more details
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 6, 2018, 6:39am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/7 "2018-09-06T06:39:54Z")
If you have local SSDs I am not sure what could be causing this. What OS and JVM are you using? What does your cluster setup and configuration look like?
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 6, 2018, 6:48am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/8 "2018-09-06T06:48:57Z")
OS:
DISTRIB\_ID=Ubuntu
DISTRIB\_RELEASE=14.04
DISTRIB\_CODENAME=trusty
DISTRIB\_DESCRIPTION="Ubuntu 14.04.5 LTS"
JVM:
openjdk version "1.8.0\_171"
OpenJDK Runtime Environment (build 1.8.0\_171-8u171-b11-2~14.04-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)
Cluster setup:
6 Master-
189 datanodes
18 ingest nodes and
3 co-ordinators
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 7, 2018, 2:47am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/9 "2018-09-07T02:47:29Z")
Hope the above info helps.
Can please share what is the usual scenarios where the chances of "shard lock" are high or can be the root cause. Anything you want to suggest how to analyse or find the root cause or how to tackle "shard lock" situation.
Or any best practice that you want to share to avoid such situation
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 10, 2018, 3:26am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/10 "2018-09-10T03:26:12Z")
@Christian_Dahlqvist any help in this case?
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 10, 2018, 5:34am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/11 "2018-09-10T05:34:59Z")
I have not come across this before, but did look through issues on the Elasticsearch GitHub repo and found [this](https://github.com/elastic/elasticsearch/issues/23199). It does however relate to older versions. It may be worthwhile looking through this and see if any of the potential causes apply to your cluster. If not, I would recommend you open a new issue with as much information about your cluster and work load as possible.
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 14, 2018, 5:17am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/12 "2018-09-14T05:17:01Z")
Thanks @Christian_Dahlqvist. The error didn't reoccur. Will raise the ticket if find one
---
### Author:  [@hzarrabi](https://discuss.elastic.co/u/hzarrabi)
#### Post date: [September 22, 2018, 10:19am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/13 "2018-09-22T10:19:32Z")
Sometimes when your are out of memory elasticsearch is blocked because it can't allocate shards.
---
### Author:  [@Priya\_Prabhakar](https://discuss.elastic.co/u/Priya_Prabhakar)
#### Post date: [September 24, 2018, 6:58am UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/14 "2018-09-24T06:58:18Z")
@hzarrabi for answering. Still unable to locate what caused the issue.
---
### Author:  [@hzarrabi](https://discuss.elastic.co/u/hzarrabi)
#### Post date: [September 24, 2018, 2:06pm UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/15 "2018-09-24T14:06:44Z")
is your elasticsearch responding by http:
[http://ipaddress:9200](http://ipaddress:9200)
if yes, do you have any indexes in red status. you can see by sending from any browser to your elastic server :
[http://ipaddress:9200/\_cat/indices?v&h=host,creation.date.string,index,health,status&s=creation.date](http://ipaddress:9200/_cat/indices?v&h=host,creation.date.string,index,health,status&s=creation.date)
if you have in red, in my case I became to resove it by deleting some red indexes from linux command line :
sudo curl XDELETE [http://ipaddress:9200/\_cat/](http://ipaddress:9200/_cat/)\
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 22, 2018, 2:06pm UTC](https://discuss.elastic.co/t/java-io-ioexception-failed-to-obtain-in-memory-shard-lock/147375/16 "2018-10-22T14:06:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.