# Debug " Request Timeout after 1500ms"
**URL:** https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017
**Category:** Elasticsearch
**Created:** [May 3, 2016, 6:55am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017 "2016-05-03T06:55:51Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 3, 2016, 6:55am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/1 "2016-05-03T06:55:51Z")
Hi,
all was working fine till few days ago. than i started to get this message: Request Timeout after 1500ms.
i can't find any info about it in the log files.
how can i debug this problem?
thanks,
guy
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 4, 2016, 5:09am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/2 "2016-05-04T05:09:36Z")
Where did you get this error?
There's not much for us in your post to go on unfortunately ☹
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 4, 2016, 9:30am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/3 "2016-05-04T09:30:14Z")
Hi,
attached the most update error massage:
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 4, 2016, 9:36am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/4 "2016-05-04T09:36:10Z")
You should look in your Elasticsearch logs at the time that this happened.
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 4, 2016, 9:42am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/5 "2016-05-04T09:42:16Z")
is that help?
at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:241)
at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:56)
at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
[2016-05-04 12:39:03,915][DEBUG][action.admin.indices.stats] [Fever Pitch] failed to execute [indices:monitor/stats] on node [s3\_zNTUiS0OeJWkVkWr40A]
RemoteTransportException[[Fever Pitch][localhost/127.0.0.1:9300][indices:monitor/stats[n]]]; nested: OutOfMemoryError[Java heap space];
Caused by: java.lang.OutOfMemoryError: Java heap space
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 4, 2016, 9:48am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/6 "2016-05-04T09:48:29Z")
this is the "top" output, how come it's Out Of Memory?
KiB Mem : 16171692 total, 10265008 free, 2295560 used, 3611124 buff/cache
KiB Swap: 8191996 total, 8191996 free, 0 used. 13473676 avail Mem
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [May 4, 2016, 10:08am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/7 "2016-05-04T10:08:41Z")
Probably because you did not allocate enough memory to elasticsearch?
You should review your settings and/or your architecture.
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 4, 2016, 11:14am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/8 "2016-05-04T11:14:02Z")
i changed the HEAP\_SIZE with the commend: export ES\_HEAP\_SIZE=10g and it solved the problem,
Thanks.
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 5, 2016, 5:05am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/9 "2016-05-05T05:05:04Z")
hi,
i was happy to soon...
at org.elasticsearch.action.bulk.TransportBulkAction$2.onFailure(TransportBulkAction.java:351)
at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:95)
at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase.finishAsFailed(TransportReplicationAction.java:567)
at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase.retry(TransportReplicationAction.java:540)
at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase$1.handleException(TransportReplicationAction.java:525)
at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:855)
at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:833)
at org.elasticsearch.transport.DelegatingTransportChannel.sendResponse(DelegatingTransportChannel.java:68)
at org.elasticsearch.transport.RequestHandlerRegistry$TransportChannelWrapper.sendResponse(RequestHandlerRegistry.java:146)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.finishAsFailed(TransportReplicationAction.java:730)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:656)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:271)
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
i will be grateful for some clue.
thanks,
guy
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [May 5, 2016, 5:42am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/10 "2016-05-05T05:42:54Z")
What do you have in elasticsearch logs? (Please format them).
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 5, 2016, 9:40am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/11 "2016-05-05T09:40:45Z")
the tail of the elasticsearch log:
java.lang.OutOfMemoryError: Java heap space
[2016-05-05 12:38:16,966][DEBUG][action.admin.indices.stats] [Chimera] [indices:monitor/stats] failed to execute operation for shard [[logstash-2016.04.15][0], node[pO0v0tCfQ922FVoXxU5Org], [P], v[58], s[STARTED], a[id=rlt48r71QPe4Dbj5RBRF-g]]
java.lang.OutOfMemoryError: Java heap space
[2016-05-05 12:38:17,070][DEBUG][action.fieldstats] [Chimera] [logstash-2016.04.01][1], node[pO0v0tCfQ922FVoXxU5Org], [P], v[66], s[STARTED], a[id=hKsdWcElThGlyPRSxOfmKg]: failed to execute [org.elasticsearch.action.fieldstats.FieldStatsRequest@607e173e]
RemoteTransportException[[Chimera][localhost/127.0.0.1:9300][indices:data/read/field\_stats[s]]]; nested: OutOfMemoryError[Java heap space];
Caused by: java.lang.OutOfMemoryError: Java heap space
[2016-05-05 12:38:21,532][INFO][index.shard] [Chimera] [logstash-2016.05.05][2] ignoring recovery of a corrupt translog entry
MapperParsingException[failed to parse]; nested: OutOfMemoryError[Java heap space];
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:154)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
at org.elasticsearch.index.shard.IndexShard.prepareCreate(IndexShard.java:529)
at org.elasticsearch.index.shard.TranslogRecoveryPerformer.performRecoveryOperation(TranslogRecoveryPerformer.java:175)
at org.elasticsearch.index.shard.TranslogRecoveryPerformer.recoveryFromSnapshot(TranslogRecoveryPerformer.java:107)
at org.elasticsearch.index.shard.IndexShard$1.recoveryFromSnapshot(IndexShard.java:1584)
at org.elasticsearch.index.engine.InternalEngine.recoverFromTranslog(InternalEngine.java:238)
at org.elasticsearch.index.engine.InternalEngine.(InternalEngine.java:174)
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25)
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:1515)
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:1499)
at org.elasticsearch.index.shard.IndexShard.internalPerformTranslogRecovery(IndexShard.java:972)
at org.elasticsearch.index.shard.IndexShard.performTranslogRecovery(IndexShard.java:944)
at org.elasticsearch.index.shard.StoreRecoveryService.recoverFromStore(StoreRecoveryService.java:241)
at org.elasticsearch.index.shard.StoreRecoveryService.access$100(StoreRecoveryService.java:56)
at org.elasticsearch.index.shard.StoreRecoveryService$1.run(StoreRecoveryService.java:129)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Java heap space
[2016-05-05 12:38:21,532][WARN][indices.ttl] [Chimera] failed to execute ttl purge
java.lang.OutOfMemoryError: Java heap space
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 5, 2016, 9:51am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/12 "2016-05-05T09:51:24Z")
Wow, lots of problems there ☹
Don't use TTL, you have a few OOMs and you also appear to have shard failures.
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 5, 2016, 10:31am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/13 "2016-05-05T10:31:53Z")
what should i do then? what is the TTL?
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 5, 2016, 1:21pm UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/14 "2016-05-05T13:21:24Z")
help, someone?
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 5, 2016, 10:17pm UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/15 "2016-05-05T22:17:26Z")
It looks like you are still having issues with memory, can you increase the heap more?
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 9, 2016, 6:21am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/16 "2016-05-09T06:21:40Z")
same results...
---
### Author:  [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [May 9, 2016, 7:45am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/17 "2016-05-09T07:45:33Z")
How much heap do you have now, how much data in the cluster, how many indices and how many shards?
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 9, 2016, 8:40am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/18 "2016-05-09T08:40:59Z")
one cluster, about 80G Data
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending\_tasks max\_task\_wait\_time active\_shards\_percent
1462783198 11:39:58 elasticsearch red **1** 1 **266** 266 0 4 277 2 1.5s 48.6%
---
### Author:  [@ggilat](https://discuss.elastic.co/u/ggilat)
#### Post date: [May 19, 2016, 11:27am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/19 "2016-05-19T11:27:03Z")
help, someone? something i can do?
---
### Author:  [@dickepa](https://discuss.elastic.co/u/dickepa)
#### Post date: [May 19, 2016, 11:31am UTC](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017/20 "2016-05-19T11:31:12Z")
Not sure if entirely relevant but have a read of [New ELK env. stopping working](https://discuss.elastic.co/t/new-elk-env-stopping-working/38251/9)
[Next page](https://discuss.elastic.co/t/debug-request-timeout-after-1500ms/49017.md?page=2)