# Large size of .hprof files on data nodes
**URL:** https://discuss.elastic.co/t/large-size-of-hprof-files-on-data-nodes/18004
**Category:** Elasticsearch
**Created:** [June 10, 2014, 6:02am UTC](https://discuss.elastic.co/t/large-size-of-hprof-files-on-data-nodes/18004 "2014-06-10T06:02:36Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Bharvi\_Dixit\_2](https://discuss.elastic.co/u/Bharvi_Dixit_2)
#### Post date: [June 10, 2014, 6:02am UTC](https://discuss.elastic.co/t/large-size-of-hprof-files-on-data-nodes/18004/1 "2014-06-10T06:02:36Z")
Hi,
I am using elasticsearch 1.0.0 & having a cluster of 7 nodes. (3 master, 2
data & 2 client nodes)
_The problem i am facing that on data nodes .hprof files are being
generated which is huge in size_ : datanode1: around 9gb & datanode2:
around 3gb
While in logs of data nodes these lines are appearing frequently:
[2014-06-09 00:10:34,442][WARN][monitor.jvm] [Server116]
[gc][young][290308][34996] duration [1.1s], collections [1]/[1.5s], total
[1.1s]/[48.1m], memory [3.6gb]-\>[3.4gb]/[7.9gb], all\_pools {[young]
[508.4mb]-\>[735.3kb]/[532.5mb]}{[survivor]
[66.5mb]-\>[66.5mb]/[66.5mb]}{[old] [3.1gb]-\>[3.3gb]/[7.3gb]}
[2014-06-09 00:19:51,533][WARN][monitor.jvm] [Server116]
[gc][young][290864][35014] duration [1.3s], collections [1]/[1.8s], total
[1.3s]/[48.2m], memory [4.1gb]-\>[4.1gb]/[7.9gb], all\_pools {[young]
[290.5mb]-\>[3.1mb]/[532.5mb]}{[survivor] [9.5mb]-\>[66.5mb]/[66.5mb]}{[old]
[3.8gb]-\>[4gb]/[7.3gb]}
The cluster contains 35 indexes with 170 gb of data (one shard & one
replica for each index).
_Is there anything wrong in java (JVM) or there is any other issue 7 what
should i do to resolve the above issues.?_
Thanks in advance,
Bharvi Dixit
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/de973225-e074-4521-bf94-36732fdd78c5%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/de973225-e074-4521-bf94-36732fdd78c5%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@Bharvi\_Dixit\_2](https://discuss.elastic.co/u/Bharvi_Dixit_2)
#### Post date: [June 10, 2014, 6:04am UTC](https://discuss.elastic.co/t/large-size-of-hprof-files-on-data-nodes/18004/2 "2014-06-10T06:04:53Z")
_On the other side, i have never got OutOfMemoryException ony any node._
On Tuesday, 10 June 2014 11:32:36 UTC+5:30, Bharvi Dixit wrote:
> Hi,
> I am using elasticsearch 1.0.0 & having a cluster of 7 nodes. (3 master, 2
> data & 2 client nodes)
> _The problem i am facing that on data nodes .hprof files are being
> generated which is huge in size_ : datanode1: around 9gb & datanode2:
> around 3gb
>
> While in logs of data nodes these lines are appearing frequently:
>
> [2014-06-09 00:10:34,442][WARN][monitor.jvm] [Server116]
> [gc][young][290308][34996] duration [1.1s], collections [1]/[1.5s], total
> [1.1s]/[48.1m], memory [3.6gb]-\>[3.4gb]/[7.9gb], all\_pools {[young]
> [508.4mb]-\>[735.3kb]/[532.5mb]}{[survivor]
> [66.5mb]-\>[66.5mb]/[66.5mb]}{[old] [3.1gb]-\>[3.3gb]/[7.3gb]}
> [2014-06-09 00:19:51,533][WARN][monitor.jvm] [Server116]
> [gc][young][290864][35014] duration [1.3s], collections [1]/[1.8s], total
> [1.3s]/[48.2m], memory [4.1gb]-\>[4.1gb]/[7.9gb], all\_pools {[young]
> [290.5mb]-\>[3.1mb]/[532.5mb]}{[survivor] [9.5mb]-\>[66.5mb]/[66.5mb]}{[old]
> [3.8gb]-\>[4gb]/[7.3gb]}
>
> The cluster contains 35 indexes with 170 gb of data (one shard & one
> replica for each index).
>
> _Is there anything wrong in java (JVM) or there is any other issue 7 what
> should i do to resolve the above issues.?_
>
> Thanks in advance,
> Bharvi Dixit
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b9997803-bfc5-4e48-bef1-4b8322f286f6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b9997803-bfc5-4e48-bef1-4b8322f286f6%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@dkota](https://discuss.elastic.co/u/dkota)
#### Post date: [July 5, 2016, 3:38pm UTC](https://discuss.elastic.co/t/large-size-of-hprof-files-on-data-nodes/18004/3 "2016-07-05T15:38:31Z")
Did you resolve this issue?
---
### Author:  [@jasontedor](https://discuss.elastic.co/u/jasontedor)
#### Post date: [July 5, 2016, 7:01pm UTC](https://discuss.elastic.co/t/large-size-of-hprof-files-on-data-nodes/18004/4 "2016-07-05T19:01:25Z")
Elasticsearch ships withs `-XX:+HeapDumpOnOutOfMemory` enabled, this leads to the heap dumps being generated whenever the JVM encounters an `OutOfMemoryError`. The Elasticsearch codebase is rife with blocks that catch `Throwable` and swallow it; this means that you might not see every instance of `OutOfMemoryError` in your logs. And an uncaught `OutOfMemoryError` will not bring down Elasticsearch anyway. This is changing though as Elasticsearch 5.0.0 will stop [catching throwable](https://github.com/elastic/elasticsearch/pull/19231) and there is an open PR to [die on `OutOfMemoryError`](https://github.com/elastic/elasticsearch/pull/19272).
If you want to stop the JVM from dumping the heap when it encounters an `OutOfMemoryError`, you need to remove the flag `-XX:+HeapDumpOnOutOfMemoryError` from the startup parameters passed to the JVM. Keep in mind though that you really want to [diagnose](https://www.elastic.co/blog/a-heap-of-trouble) why Elasticsearch is running out of memory, and heap dumps might be useful for doing that.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:37pm UTC](https://discuss.elastic.co/t/large-size-of-hprof-files-on-data-nodes/18004/5 "2017-07-05T22:37:49Z")